Oct 29 23:10:20.321952 ip-10-0-130-86 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Oct 29 23:10:20.321963 ip-10-0-130-86 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Oct 29 23:10:20.321973 ip-10-0-130-86 systemd[1]: kubelet.service: Failed with result 'resources'. Oct 29 23:10:20.322348 ip-10-0-130-86 systemd[1]: Failed to start Kubernetes Kubelet. Oct 29 23:10:29.628786 ip-10-0-130-86 systemd[1]: kubelet.service: Failed to schedule restart job: Unit openvswitch.service not found. Oct 29 23:10:29.628801 ip-10-0-130-86 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot d1d15794869e496db1b6185e9854df6a -- Oct 29 23:12:09.870394 ip-10-0-130-86 systemd[1]: Starting Kubernetes Kubelet... Oct 29 23:12:10.324517 ip-10-0-130-86 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 29 23:12:10.324517 ip-10-0-130-86 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 29 23:12:10.324517 ip-10-0-130-86 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 29 23:12:10.324517 ip-10-0-130-86 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 29 23:12:10.329847 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329765 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 23:12:10.329847 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329846 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 23:12:10.329847 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329851 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329854 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329857 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329860 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329864 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329867 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329869 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329872 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329875 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329878 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329882 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329887 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329890 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329893 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329896 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329899 2575 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329901 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329904 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329907 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329909 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 23:12:10.329941 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329912 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329914 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329917 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329920 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329922 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329928 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329931 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329934 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329937 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329939 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329942 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329945 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329947 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329950 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329953 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329956 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329959 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329962 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329964 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329967 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 23:12:10.330412 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329970 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329972 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329975 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329977 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329980 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329984 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329988 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329991 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329993 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329997 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.329999 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330002 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330004 2575 feature_gate.go:386] unrecognized feature gate: Example Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330007 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330010 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330013 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330016 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330018 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330021 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 23:12:10.330880 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330024 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330026 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330029 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330032 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330034 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330037 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330040 2575 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330042 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330045 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330047 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330050 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330052 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330055 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330057 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330060 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330062 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330065 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330068 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330070 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 23:12:10.331338 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330073 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330075 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330078 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330081 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330084 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330087 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330089 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330092 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330095 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330097 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330100 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330103 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330118 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330121 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330123 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330126 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330129 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330131 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330135 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 23:12:10.331828 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330138 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330140 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330143 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330146 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330148 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330151 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330153 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330936 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330943 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330947 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330950 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330953 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330956 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330959 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330961 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330964 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330967 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330970 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330973 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330977 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 23:12:10.332290 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330981 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330985 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330988 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330990 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330993 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330996 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.330999 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331001 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331004 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331007 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331009 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331012 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331014 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331017 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331020 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331023 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331025 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331028 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331038 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331042 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 23:12:10.333716 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331045 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331048 2575 feature_gate.go:386] unrecognized feature gate: Example Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331051 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331054 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331056 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331059 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331061 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331064 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331067 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331070 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331072 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331075 2575 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331077 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331081 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331084 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331086 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331089 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331092 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331094 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331097 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 23:12:10.334205 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331099 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331102 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331119 2575 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331122 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331125 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331128 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331131 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331134 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331136 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331140 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331142 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331145 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331148 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331150 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331154 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331157 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331161 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331164 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331167 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 23:12:10.334670 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331170 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331174 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331178 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331180 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331183 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331186 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331189 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331191 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331194 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331196 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331199 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331202 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331204 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331207 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331209 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331212 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331215 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331217 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331220 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 23:12:10.335130 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331222 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331225 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331227 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331230 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331232 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331235 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331237 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331240 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331242 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331245 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331247 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331250 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331253 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331256 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331259 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331741 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331747 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331750 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331753 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 23:12:10.335583 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331756 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331759 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331761 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331764 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331766 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331769 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331772 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331775 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331777 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331780 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331782 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331785 2575 feature_gate.go:386] unrecognized feature gate: Example Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331788 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331790 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331793 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331796 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331799 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331801 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331804 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331806 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 23:12:10.336032 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331809 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331812 2575 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331815 2575 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331818 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331820 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331823 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331826 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331828 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331832 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331836 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331839 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331842 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331845 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331847 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331850 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331852 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331855 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331857 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331860 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331862 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 23:12:10.336511 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331864 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331867 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331870 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331872 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331875 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331877 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331880 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331883 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331886 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331889 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331892 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331895 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331898 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331900 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331903 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331906 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331909 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331912 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331914 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 23:12:10.337009 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331917 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331919 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331922 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331924 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331927 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331929 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331932 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331935 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331938 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331940 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331943 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331945 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331948 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331950 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331952 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331955 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331957 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331960 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331962 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 23:12:10.337541 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331965 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331968 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331971 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331974 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331976 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331979 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331981 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331984 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331987 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331989 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331992 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331994 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331997 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.331999 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.332002 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.332005 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.332007 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.332010 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.332012 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.332015 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 23:12:10.337991 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.332017 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.332020 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.332022 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.332025 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333225 2575 flags.go:64] FLAG: --address="0.0.0.0" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333237 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333244 2575 flags.go:64] FLAG: --anonymous-auth="true" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333249 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333253 2575 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333256 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333261 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333266 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333269 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333272 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333275 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333278 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333282 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333285 2575 flags.go:64] FLAG: --cgroup-root="" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333288 2575 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333292 2575 flags.go:64] FLAG: --client-ca-file="" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333294 2575 flags.go:64] FLAG: --cloud-provider="external" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333297 2575 flags.go:64] FLAG: --cluster-dns="[]" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333302 2575 flags.go:64] FLAG: --cluster-domain="" Oct 29 23:12:10.338491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333305 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333308 2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333312 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333316 2575 flags.go:64] FLAG: --container-log-max-files="5" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333320 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333324 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333327 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333330 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333333 2575 flags.go:64] FLAG: --contention-profiling="false" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333336 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333339 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333343 2575 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333346 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333350 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333354 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333357 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333360 2575 flags.go:64] FLAG: --enable-load-reader="false" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333363 2575 flags.go:64] FLAG: --enable-server="true" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333366 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333370 2575 flags.go:64] FLAG: --event-burst="100" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333374 2575 flags.go:64] FLAG: --event-qps="50" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333376 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333379 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333382 2575 flags.go:64] FLAG: --eviction-hard="" Oct 29 23:12:10.339100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333387 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333390 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333393 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333396 2575 flags.go:64] FLAG: --eviction-soft="" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333399 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333402 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333405 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333408 2575 flags.go:64] FLAG: --experimental-mounter-path="" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333411 2575 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333414 2575 flags.go:64] FLAG: --fail-swap-on="true" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333417 2575 flags.go:64] FLAG: --feature-gates="" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333422 2575 flags.go:64] FLAG: --file-check-frequency="20s" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333425 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333428 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333431 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333435 2575 flags.go:64] FLAG: --healthz-port="10248" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333438 2575 flags.go:64] FLAG: --help="false" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333441 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-130-86.ec2.internal" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333444 2575 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333448 2575 flags.go:64] FLAG: --http-check-frequency="20s" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333451 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333454 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333460 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333463 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 29 23:12:10.339771 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333465 2575 flags.go:64] FLAG: --image-service-endpoint="" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333468 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333471 2575 flags.go:64] FLAG: --kube-api-burst="100" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333474 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333477 2575 flags.go:64] FLAG: --kube-api-qps="50" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333480 2575 flags.go:64] FLAG: --kube-reserved="" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333483 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333486 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333489 2575 flags.go:64] FLAG: --kubelet-cgroups="" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333492 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333495 2575 flags.go:64] FLAG: --lock-file="" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333498 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333502 2575 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333505 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333511 2575 flags.go:64] FLAG: --log-json-split-stream="false" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333514 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333517 2575 flags.go:64] FLAG: --log-text-split-stream="false" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333520 2575 flags.go:64] FLAG: --logging-format="text" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333522 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333526 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333529 2575 flags.go:64] FLAG: --manifest-url="" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333532 2575 flags.go:64] FLAG: --manifest-url-header="" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333536 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333539 2575 flags.go:64] FLAG: --max-open-files="1000000" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333543 2575 flags.go:64] FLAG: --max-pods="110" Oct 29 23:12:10.340431 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333546 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333549 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333552 2575 flags.go:64] FLAG: --memory-manager-policy="None" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333556 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333559 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333563 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333566 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333574 2575 flags.go:64] FLAG: --node-status-max-images="50" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333577 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333580 2575 flags.go:64] FLAG: --oom-score-adj="-999" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333583 2575 flags.go:64] FLAG: --pod-cidr="" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333587 2575 flags.go:64] FLAG: --pod-manifest-path="" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333590 2575 flags.go:64] FLAG: --pod-max-pids="-1" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333593 2575 flags.go:64] FLAG: --pods-per-core="0" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333596 2575 flags.go:64] FLAG: --port="10250" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333599 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333602 2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-056125f30a3a4f1a1" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333605 2575 flags.go:64] FLAG: --qos-reserved="" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333608 2575 flags.go:64] FLAG: --read-only-port="10255" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333611 2575 flags.go:64] FLAG: --register-node="true" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333614 2575 flags.go:64] FLAG: --register-with-taints="" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333618 2575 flags.go:64] FLAG: --registry-burst="10" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333621 2575 flags.go:64] FLAG: --registry-qps="5" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333624 2575 flags.go:64] FLAG: --reserved-cpus="" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333627 2575 flags.go:64] FLAG: --reserved-memory="" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333631 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 29 23:12:10.341450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333635 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333637 2575 flags.go:64] FLAG: --rotate-certificates="false" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333640 2575 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333643 2575 flags.go:64] FLAG: --runonce="false" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333646 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333649 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333652 2575 flags.go:64] FLAG: --seccomp-default="false" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333655 2575 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333657 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333661 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333665 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333669 2575 flags.go:64] FLAG: --storage-driver-password="root" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333672 2575 flags.go:64] FLAG: --storage-driver-secure="false" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333675 2575 flags.go:64] FLAG: --storage-driver-table="stats" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333678 2575 flags.go:64] FLAG: --storage-driver-user="root" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333681 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333684 2575 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333687 2575 flags.go:64] FLAG: --system-cgroups="" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333690 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333695 2575 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333702 2575 flags.go:64] FLAG: --tls-cert-file="" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333705 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333709 2575 flags.go:64] FLAG: --tls-min-version="" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333712 2575 flags.go:64] FLAG: --tls-private-key-file="" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333715 2575 flags.go:64] FLAG: --topology-manager-policy="none" Oct 29 23:12:10.342224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333718 2575 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333721 2575 flags.go:64] FLAG: --topology-manager-scope="container" Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333725 2575 flags.go:64] FLAG: --v="2" Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333729 2575 flags.go:64] FLAG: --version="false" Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333733 2575 flags.go:64] FLAG: --vmodule="" Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333738 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.333741 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333873 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333877 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333881 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333884 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333887 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333890 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333892 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333895 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333897 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333900 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333902 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333907 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333909 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 23:12:10.342812 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333912 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333914 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333917 2575 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333919 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333922 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333925 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333927 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333931 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333934 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333936 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333939 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333942 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333945 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333948 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333950 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333953 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333955 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333958 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333960 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333964 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 23:12:10.343345 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333968 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333971 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333974 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333977 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333980 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333983 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333986 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333988 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333991 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333994 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.333997 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334001 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334004 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334006 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334009 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334012 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334016 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334019 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334022 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 23:12:10.343835 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334026 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334028 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334031 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334034 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334037 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334039 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334042 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334044 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334047 2575 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334050 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334052 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334055 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334057 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334060 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334063 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334066 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334068 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334071 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334073 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 23:12:10.344289 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334076 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334078 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334081 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334083 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334087 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334090 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334093 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334096 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334098 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334100 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334103 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334123 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334125 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334129 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334132 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334135 2575 feature_gate.go:386] unrecognized feature gate: Example Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334137 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334140 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334143 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334145 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 23:12:10.344740 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334148 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 23:12:10.345232 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334150 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 23:12:10.345232 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334153 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 23:12:10.345232 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334156 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 23:12:10.345232 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334158 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 23:12:10.345232 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334161 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 23:12:10.345232 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334163 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 23:12:10.345232 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334166 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 23:12:10.345232 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334169 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 23:12:10.345232 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334171 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 23:12:10.345232 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334174 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 23:12:10.345232 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334176 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 23:12:10.345232 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334179 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 23:12:10.345232 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334182 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 23:12:10.345232 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334185 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 23:12:10.345232 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.334212 2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 29 23:12:10.345232 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334248 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 23:12:10.345232 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334253 2575 feature_gate.go:386] unrecognized feature gate: Example Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334256 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334259 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334262 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334265 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334267 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334270 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334272 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334275 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334279 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334282 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334284 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334287 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334290 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334292 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334295 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334297 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334300 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334303 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334306 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 23:12:10.345626 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334309 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334311 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334315 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334317 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334320 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334323 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334325 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334328 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334330 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334333 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334335 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334338 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334341 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334343 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334346 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334349 2575 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334352 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334354 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334357 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 23:12:10.346100 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334360 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334362 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334366 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334368 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334371 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334373 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334377 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334379 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334382 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334384 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334387 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334389 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334392 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334396 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334399 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334402 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334405 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334407 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334410 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334412 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 23:12:10.346651 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334415 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334417 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334420 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334422 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334425 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334428 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334430 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334433 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334435 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334440 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334443 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334446 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334449 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334452 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334456 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334459 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334461 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334464 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334466 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 23:12:10.347341 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334469 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334471 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334474 2575 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334476 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334479 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334482 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334484 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334487 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334489 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334492 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334495 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334497 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334500 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334502 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334505 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334507 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334510 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334513 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334515 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 23:12:10.347802 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334518 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334520 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334523 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334525 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334528 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334531 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334534 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.334537 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.342575 2575 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.342593 2575 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.342611 2575 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.342616 2575 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342674 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342679 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342683 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342686 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342689 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342692 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342695 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 23:12:10.348258 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342698 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342700 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342703 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342705 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342708 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342711 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342713 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342716 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342719 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342722 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342724 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342727 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342730 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342733 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342736 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342739 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342750 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342753 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342755 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 23:12:10.348695 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342758 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342761 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342763 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342766 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342769 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342772 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342774 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342777 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342779 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342782 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342784 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342787 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342790 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342792 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342795 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342798 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342800 2575 feature_gate.go:386] unrecognized feature gate: Example Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342803 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342805 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342808 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 23:12:10.349145 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342810 2575 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342813 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342816 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342819 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342822 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342825 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342827 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342830 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342833 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342836 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342838 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342841 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342843 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342846 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342849 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342851 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342854 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342856 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342859 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342861 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 23:12:10.349611 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342864 2575 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342866 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342870 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342875 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342878 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342881 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342884 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342887 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342890 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342892 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342895 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342899 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342902 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342906 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342909 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342911 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342914 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342916 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342919 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 23:12:10.350075 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342922 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342924 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342927 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342930 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342932 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342935 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342938 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342940 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342943 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342945 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342948 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342951 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342953 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342956 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342958 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342961 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342963 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342966 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342968 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 23:12:10.350543 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342971 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 23:12:10.350993 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.342973 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 23:12:10.350993 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.343000 2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 29 23:12:10.350993 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343154 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 23:12:10.350993 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343158 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 23:12:10.350993 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343162 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 23:12:10.350993 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343164 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 23:12:10.350993 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343167 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 23:12:10.350993 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343171 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 23:12:10.350993 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343173 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 23:12:10.350993 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343176 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 23:12:10.350993 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343178 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 23:12:10.350993 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343181 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 23:12:10.350993 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343184 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 23:12:10.350993 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343187 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 23:12:10.350993 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343192 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343195 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343198 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343201 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343203 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343206 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343208 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343211 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343213 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343216 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343219 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343221 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343224 2575 feature_gate.go:386] unrecognized feature gate: Example Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343226 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343229 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343232 2575 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343234 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343236 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343239 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343242 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 23:12:10.351366 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343245 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343247 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343250 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343252 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343255 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343257 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343261 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343263 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343266 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343268 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343271 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343274 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343276 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343279 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343281 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343284 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343286 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343289 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343291 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343294 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 23:12:10.351842 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343296 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343298 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343301 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343303 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343306 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343308 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343311 2575 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343313 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343316 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343319 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343321 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343324 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343327 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343329 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343332 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343334 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343337 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343340 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343342 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343346 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 23:12:10.352312 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343349 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343352 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343354 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343357 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343360 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343363 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343365 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343368 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343370 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343373 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343378 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343381 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343385 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343388 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343391 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343394 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343396 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343401 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343404 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 23:12:10.352786 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343406 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 23:12:10.353262 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343409 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 23:12:10.353262 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343412 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 23:12:10.353262 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343414 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 23:12:10.353262 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343417 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 23:12:10.353262 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343419 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 23:12:10.353262 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343422 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 23:12:10.353262 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343425 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 23:12:10.353262 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343427 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 23:12:10.353262 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343431 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 23:12:10.353262 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343433 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 23:12:10.353262 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343436 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 23:12:10.353262 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343439 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 23:12:10.353262 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343441 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 23:12:10.353262 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343444 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 23:12:10.353262 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.343467 2575 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343498 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343502 2575 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343505 2575 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343508 2575 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343510 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343513 2575 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343515 2575 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343518 2575 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343521 2575 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343523 2575 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343526 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343528 2575 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343531 2575 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343533 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343537 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343540 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343543 2575 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343545 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343548 2575 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 23:12:10.353637 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343551 2575 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343553 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343556 2575 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343558 2575 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343561 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343563 2575 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343566 2575 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343569 2575 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343572 2575 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343574 2575 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343577 2575 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343580 2575 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343583 2575 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343587 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343590 2575 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343593 2575 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343596 2575 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343598 2575 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343601 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 23:12:10.354096 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343604 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343606 2575 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343609 2575 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343611 2575 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343613 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343616 2575 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343618 2575 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343621 2575 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343625 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343627 2575 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343630 2575 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343633 2575 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343635 2575 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343638 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343640 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343643 2575 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343645 2575 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343648 2575 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343651 2575 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 23:12:10.354540 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343653 2575 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343656 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343658 2575 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343661 2575 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343663 2575 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343666 2575 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343669 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343673 2575 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343676 2575 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343678 2575 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343681 2575 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343684 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343686 2575 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343689 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343691 2575 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343694 2575 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343696 2575 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343699 2575 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343701 2575 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 23:12:10.355015 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343704 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343706 2575 feature_gate.go:386] unrecognized feature gate: Example Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343710 2575 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343713 2575 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343715 2575 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343718 2575 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343727 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343731 2575 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343734 2575 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343736 2575 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343739 2575 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343741 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343744 2575 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343747 2575 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343749 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343752 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343754 2575 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343757 2575 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343759 2575 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343762 2575 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343766 2575 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 23:12:10.355519 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343769 2575 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 23:12:10.356016 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343772 2575 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 23:12:10.356016 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343774 2575 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 23:12:10.356016 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343777 2575 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 23:12:10.356016 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343780 2575 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 23:12:10.356016 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343782 2575 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 23:12:10.356016 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343784 2575 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 23:12:10.356016 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343787 2575 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 23:12:10.356016 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:10.343790 2575 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 23:12:10.356016 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.343850 2575 server.go:957] "Client rotation is on, will bootstrap in background" Oct 29 23:12:10.356016 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.346250 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Oct 29 23:12:10.356016 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.347318 2575 server.go:1014] "Starting client certificate rotation" Oct 29 23:12:10.356016 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.347416 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 29 23:12:10.356016 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.347456 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 29 23:12:10.356016 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.350385 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 29 23:12:10.356016 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.351494 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 29 23:12:10.361514 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.361495 2575 log.go:25] "Validated CRI v1 runtime API" Oct 29 23:12:10.365574 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.365561 2575 log.go:25] "Validated CRI v1 image API" Oct 29 23:12:10.367101 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.367079 2575 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 29 23:12:10.372220 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.372190 2575 fs.go:135] Filesystem UUIDs: map[2c71c32d-ebcc-40e6-8ad9-5114d692bd30:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f6ddfd90-f75a-4f7c-84b2-7a0b35e091af:/dev/nvme0n1p4] Oct 29 23:12:10.372296 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.372217 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Oct 29 23:12:10.378664 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.378544 2575 manager.go:217] Machine: {Timestamp:2026-10-29 23:12:10.378052858 +0000 UTC m=+0.376929788 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097870 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec265682fb0b989e8fef09d21f55ab99 SystemUUID:ec265682-fb0b-989e-8fef-09d21f55ab99 BootID:d1d15794-869e-496d-b1b6-185e9854df6a Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:c0:a6:21:ab Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:c0:a6:21:ab Speed:0 Mtu:9001} {Name:ovs-system MacAddress:4e:72:db:26:c4:80 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 29 23:12:10.378664 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.378660 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 29 23:12:10.378801 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.378740 2575 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 29 23:12:10.380000 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.379975 2575 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 29 23:12:10.380145 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.380001 2575 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-86.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 29 23:12:10.380187 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.380147 2575 topology_manager.go:143] "Creating topology manager with none policy" Oct 29 23:12:10.380187 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.380156 2575 container_manager_linux.go:311] "Creating device plugin manager" Oct 29 23:12:10.380187 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.380186 2575 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 29 23:12:10.380267 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.380207 2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 29 23:12:10.381450 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.381440 2575 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 29 23:12:10.386748 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.386735 2575 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 29 23:12:10.386867 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.386857 2575 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 29 23:12:10.387704 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.387695 2575 kubelet.go:493] "Attempting to sync node with API server" Oct 29 23:12:10.387749 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.387708 2575 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 29 23:12:10.387749 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.387718 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 29 23:12:10.387749 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.387734 2575 kubelet.go:405] "Adding apiserver pod source" Oct 29 23:12:10.387749 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.387743 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 29 23:12:10.388143 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.388125 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 29 23:12:10.390609 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.390595 2575 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 29 23:12:10.397230 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.397218 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 29 23:12:10.398079 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.398066 2575 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 29 23:12:10.398128 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.398098 2575 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 29 23:12:10.398274 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.398263 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 29 23:12:10.398302 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.398282 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 29 23:12:10.398302 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.398290 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 29 23:12:10.398302 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.398295 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 29 23:12:10.398302 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.398301 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 29 23:12:10.398406 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.398306 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 29 23:12:10.399085 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.399077 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 29 23:12:10.399134 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.399086 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 29 23:12:10.399134 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.399094 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 29 23:12:10.399134 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.399101 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 29 23:12:10.399134 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.399126 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 29 23:12:10.399134 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.399135 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 29 23:12:10.400487 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.400478 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 29 23:12:10.400487 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.400487 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 29 23:12:10.404406 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.404394 2575 server.go:1263] "Started kubelet" Oct 29 23:12:10.404537 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.404466 2575 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 29 23:12:10.404697 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.404579 2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 29 23:12:10.404768 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.404757 2575 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 29 23:12:10.406369 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.406338 2575 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 29 23:12:10.406785 ip-10-0-130-86 systemd[1]: Started Kubernetes Kubelet. Oct 29 23:12:10.408715 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.408693 2575 server.go:317] "Adding debug handlers to kubelet server" Oct 29 23:12:10.411733 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.411694 2575 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Oct 29 23:12:10.414615 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.414597 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 29 23:12:10.415916 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.415902 2575 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 29 23:12:10.416045 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.416023 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:10.416704 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.416685 2575 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 29 23:12:10.416788 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.416707 2575 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 29 23:12:10.416788 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.416686 2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 29 23:12:10.416788 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.416784 2575 reconstruct.go:97] "Volume reconstruction finished" Oct 29 23:12:10.416788 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.416790 2575 reconciler.go:29] "Reconciler: start to sync state" Oct 29 23:12:10.416940 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.416802 2575 factory.go:55] Registering systemd factory Oct 29 23:12:10.416940 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.416816 2575 factory.go:223] Registration of the systemd container factory successfully Oct 29 23:12:10.417061 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.417039 2575 factory.go:153] Registering CRI-O factory Oct 29 23:12:10.417061 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.417056 2575 factory.go:223] Registration of the crio container factory successfully Oct 29 23:12:10.417190 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.417135 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 29 23:12:10.417190 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.417157 2575 factory.go:103] Registering Raw factory Oct 29 23:12:10.417190 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.417168 2575 manager.go:1199] Started watching for new ooms in manager Oct 29 23:12:10.418252 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.418235 2575 manager.go:319] Starting recovery of all containers Oct 29 23:12:10.429687 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.429591 2575 manager.go:324] Recovery completed Oct 29 23:12:10.434557 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.434544 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 23:12:10.442137 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.441746 2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-86.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Oct 29 23:12:10.442137 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.441913 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientMemory" Oct 29 23:12:10.442137 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.441960 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 23:12:10.442137 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.441986 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientPID" Oct 29 23:12:10.442137 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.442055 2575 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-86.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Oct 29 23:12:10.442626 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.442608 2575 cpu_manager.go:226] "Starting" policy="none" Oct 29 23:12:10.442626 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.442617 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 23:12:10.442626 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.442622 2575 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 29 23:12:10.442798 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.442638 2575 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 29 23:12:10.442947 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.441748 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-86.ec2.internal.18e322f937f17cdf default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-86.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-86.ec2.internal,},FirstTimestamp:2026-10-29 23:12:10.404363487 +0000 UTC m=+0.403240415,LastTimestamp:2026-10-29 23:12:10.404363487 +0000 UTC m=+0.403240415,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-86.ec2.internal,}" Oct 29 23:12:10.443420 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.443394 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientMemory" Oct 29 23:12:10.443484 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.443433 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 23:12:10.443484 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.443447 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientPID" Oct 29 23:12:10.444821 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.444807 2575 policy_none.go:50] "Start" Oct 29 23:12:10.444875 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.444828 2575 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 29 23:12:10.444875 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.444838 2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 29 23:12:10.446857 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.446844 2575 policy_none.go:44] "Start" Oct 29 23:12:10.450337 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.450317 2575 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 29 23:12:10.458952 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.458931 2575 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Oct 29 23:12:10.459158 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.459048 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-86.ec2.internal.18e322f93a2eea7e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-86.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-86.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-86.ec2.internal,},FirstTimestamp:2026-10-29 23:12:10.441943678 +0000 UTC m=+0.440820614,LastTimestamp:2026-10-29 23:12:10.441943678 +0000 UTC m=+0.440820614,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-86.ec2.internal,}" Oct 29 23:12:10.470662 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.470637 2575 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Oct 29 23:12:10.470754 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.470697 2575 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-86.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Oct 29 23:12:10.471280 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.471232 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pkqgq" Oct 29 23:12:10.583887 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.482956 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-pkqgq" Oct 29 23:12:10.583887 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.484781 2575 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 29 23:12:10.583887 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.490398 2575 manager.go:341] "Starting Device Plugin manager" Oct 29 23:12:10.583887 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.490419 2575 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 29 23:12:10.583887 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.490426 2575 server.go:87] "Starting device plugin registration server" Oct 29 23:12:10.583887 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.490634 2575 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 29 23:12:10.583887 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.490643 2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 29 23:12:10.583887 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.490742 2575 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 29 23:12:10.583887 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.490819 2575 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 29 23:12:10.583887 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.490827 2575 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 29 23:12:10.583887 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.491572 2575 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 29 23:12:10.583887 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.491620 2575 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:10.583887 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.492276 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-86.ec2.internal.18e322f93a2f459d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-86.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-86.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-86.ec2.internal,},FirstTimestamp:2026-10-29 23:12:10.441967005 +0000 UTC m=+0.440843935,LastTimestamp:2026-10-29 23:12:10.441967005 +0000 UTC m=+0.440843935,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-86.ec2.internal,}" Oct 29 23:12:10.583887 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.509639 2575 container_manager_linux.go:682] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2611 into the system container \"/system.slice\": " Oct 29 23:12:10.583887 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.545689 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-86.ec2.internal.18e322f93a2fabb8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-86.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-86.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-86.ec2.internal,},FirstTimestamp:2026-10-29 23:12:10.441993144 +0000 UTC m=+0.440870080,LastTimestamp:2026-10-29 23:12:10.441993144 +0000 UTC m=+0.440870080,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-86.ec2.internal,}" Oct 29 23:12:10.591497 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.591479 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 23:12:10.592767 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.592740 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientMemory" Oct 29 23:12:10.592860 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.592773 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 23:12:10.592860 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.592784 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientPID" Oct 29 23:12:10.592860 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.592811 2575 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-86.ec2.internal" Oct 29 23:12:10.593198 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.593088 2575 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-86.ec2.internal.18e322f93a2eea7e\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-86.ec2.internal.18e322f93a2eea7e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-86.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-86.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-86.ec2.internal,},FirstTimestamp:2026-10-29 23:12:10.441943678 +0000 UTC m=+0.440820614,LastTimestamp:2026-10-29 23:12:10.443418686 +0000 UTC m=+0.442295617,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-86.ec2.internal,}" Oct 29 23:12:10.597746 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.597719 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 29 23:12:10.598862 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.598840 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 29 23:12:10.598929 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.598868 2575 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 29 23:12:10.598929 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.598891 2575 kubelet.go:2521] "Starting kubelet main sync loop" Oct 29 23:12:10.599003 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.598932 2575 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Oct 29 23:12:10.647693 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.647673 2575 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 23:12:10.655739 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.655716 2575 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-86.ec2.internal" Oct 29 23:12:10.655788 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.655747 2575 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-86.ec2.internal\": node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:10.697979 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.697957 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:10.700024 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.700007 2575 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal"] Oct 29 23:12:10.700101 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.700058 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 23:12:10.700915 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.700897 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientMemory" Oct 29 23:12:10.701005 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.700924 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 23:12:10.701005 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.700933 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientPID" Oct 29 23:12:10.703200 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.703188 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 23:12:10.703345 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.703332 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:10.703382 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.703361 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 23:12:10.704360 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.704346 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientMemory" Oct 29 23:12:10.704421 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.704373 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 23:12:10.704421 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.704382 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientPID" Oct 29 23:12:10.704481 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.704346 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientMemory" Oct 29 23:12:10.704481 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.704446 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 23:12:10.704481 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.704456 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientPID" Oct 29 23:12:10.706514 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.706499 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" Oct 29 23:12:10.706585 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.706522 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 23:12:10.707499 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.707482 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientMemory" Oct 29 23:12:10.707580 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.707511 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 23:12:10.707580 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.707527 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientPID" Oct 29 23:12:10.718334 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.718315 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/966fb6b19f49ae6dfc8b44441b931f34-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal\" (UID: \"966fb6b19f49ae6dfc8b44441b931f34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:10.718439 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.718341 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/47e5a52938e8a36be364af182a627291-config\") pod \"kube-apiserver-proxy-ip-10-0-130-86.ec2.internal\" (UID: \"47e5a52938e8a36be364af182a627291\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" Oct 29 23:12:10.718439 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.718364 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/966fb6b19f49ae6dfc8b44441b931f34-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal\" (UID: \"966fb6b19f49ae6dfc8b44441b931f34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:10.735792 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.735776 2575 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-86.ec2.internal\" not found" node="ip-10-0-130-86.ec2.internal" Oct 29 23:12:10.737755 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.737740 2575 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-86.ec2.internal\" not found" node="ip-10-0-130-86.ec2.internal" Oct 29 23:12:10.798664 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.798650 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:10.818877 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.818859 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/966fb6b19f49ae6dfc8b44441b931f34-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal\" (UID: \"966fb6b19f49ae6dfc8b44441b931f34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:10.818958 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.818884 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/966fb6b19f49ae6dfc8b44441b931f34-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal\" (UID: \"966fb6b19f49ae6dfc8b44441b931f34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:10.818958 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.818900 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/47e5a52938e8a36be364af182a627291-config\") pod \"kube-apiserver-proxy-ip-10-0-130-86.ec2.internal\" (UID: \"47e5a52938e8a36be364af182a627291\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" Oct 29 23:12:10.818958 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.818934 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/47e5a52938e8a36be364af182a627291-config\") pod \"kube-apiserver-proxy-ip-10-0-130-86.ec2.internal\" (UID: \"47e5a52938e8a36be364af182a627291\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" Oct 29 23:12:10.818958 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.818953 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/966fb6b19f49ae6dfc8b44441b931f34-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal\" (UID: \"966fb6b19f49ae6dfc8b44441b931f34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:10.819079 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:10.818959 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/966fb6b19f49ae6dfc8b44441b931f34-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal\" (UID: \"966fb6b19f49ae6dfc8b44441b931f34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:10.899395 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.899335 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:10.999425 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:10.999399 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:11.040988 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:11.040963 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:12.503315 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:12.503293 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" Oct 29 23:12:12.560419 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:12.560388 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:12.661010 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:12.660942 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:12.761463 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:12.761422 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:12.807086 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:12.807061 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Oct 29 23:12:12.807578 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:12.807186 2575 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Oct 29 23:12:12.862315 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:12.862282 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:12.905366 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:12.905333 2575 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 23:12:12.962457 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:12.962386 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:12.962457 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:12.962403 2575 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 23:12:12.964558 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:12.964539 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientMemory" Oct 29 23:12:12.964707 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:12.964577 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 23:12:12.964707 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:12.964588 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientPID" Oct 29 23:12:13.043597 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.043539 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-28 23:07:11 +0000 UTC" deadline="2028-06-11 21:53:08.243950079 +0000 UTC" Oct 29 23:12:13.043597 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.043590 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14182h40m55.200364053s" Oct 29 23:12:13.062672 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:13.062650 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:13.098008 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:13.097972 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47e5a52938e8a36be364af182a627291.slice/crio-579027c84fc7add6a266a6a9736f729b2d15b74fc0c1f2f9317ffdd6d25c5d9d WatchSource:0}: Error finding container 579027c84fc7add6a266a6a9736f729b2d15b74fc0c1f2f9317ffdd6d25c5d9d: Status 404 returned error can't find the container with id 579027c84fc7add6a266a6a9736f729b2d15b74fc0c1f2f9317ffdd6d25c5d9d Oct 29 23:12:13.098414 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:13.098393 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod966fb6b19f49ae6dfc8b44441b931f34.slice/crio-f4267b48ecd99c04495ab3c0a8971ee0931c33620e4b6c0e69d7338f0365f59a WatchSource:0}: Error finding container f4267b48ecd99c04495ab3c0a8971ee0931c33620e4b6c0e69d7338f0365f59a: Status 404 returned error can't find the container with id f4267b48ecd99c04495ab3c0a8971ee0931c33620e4b6c0e69d7338f0365f59a Oct 29 23:12:13.104705 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.104690 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 29 23:12:13.163555 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:13.163530 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:13.197368 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.197346 2575 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 23:12:13.264490 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:13.264424 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:13.365338 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:13.365304 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:13.465860 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:13.465830 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:13.566492 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:13.566413 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:13.666916 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:13.666880 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:13.767430 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:13.767394 2575 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:13.824033 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.823958 2575 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 23:12:13.849243 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.849206 2575 apiserver.go:52] "Watching apiserver" Oct 29 23:12:13.875773 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.875746 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Oct 29 23:12:13.876333 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.876313 2575 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" Oct 29 23:12:13.892477 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.892450 2575 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 29 23:12:13.892805 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.892786 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-wd9cn","openshift-dns/node-resolver-zr8dl","openshift-image-registry/node-ca-jg249","openshift-ovn-kubernetes/ovnkube-node-blg76","kube-system/global-pull-secret-syncer-gz4mg","openshift-multus/multus-49s6q","openshift-multus/multus-additional-cni-plugins-xzb6b","openshift-network-operator/iptables-alerter-c9pz5","kube-system/konnectivity-agent-25m4p","openshift-network-diagnostics/network-check-target-dn96d","openshift-multus/network-metrics-daemon-2xktq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw"] Oct 29 23:12:13.895367 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.895344 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:13.898389 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.898368 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:13.900465 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.900448 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:13.902522 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.902502 2575 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 29 23:12:13.902597 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.902584 2575 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:13.904448 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.903142 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.906026 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.906004 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:13.906116 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:13.906081 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gz4mg" podUID="cf41b924-24b7-4a89-9d11-af8b4246fcde" Oct 29 23:12:13.908274 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.908241 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-49s6q" Oct 29 23:12:13.910545 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.910525 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:13.911408 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.911391 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kcr8p\"" Oct 29 23:12:13.911492 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.911420 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 29 23:12:13.911492 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.911458 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 29 23:12:13.912953 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.912936 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:13.913780 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.913761 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7fjkb\"" Oct 29 23:12:13.913780 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.913777 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 29 23:12:13.913913 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.913867 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 29 23:12:13.915678 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.915661 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 29 23:12:13.915805 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.915789 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-25m4p" Oct 29 23:12:13.915889 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.915832 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w5p5k\"" Oct 29 23:12:13.918351 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.918334 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:13.918437 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:13.918384 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dn96d" podUID="b368f5ac-f18a-49d2-b9be-6cca37b7505b" Oct 29 23:12:13.920711 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.920697 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 29 23:12:13.920999 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.920859 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 29 23:12:13.921064 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.921035 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 29 23:12:13.921064 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.920723 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 29 23:12:13.921163 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.921147 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vsvpd\"" Oct 29 23:12:13.921324 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.921307 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 29 23:12:13.921399 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.921331 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:13.921455 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:13.921397 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xktq" podUID="c6fc8782-2d71-4cb3-ab4b-b1d14aad460f" Oct 29 23:12:13.922462 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.922430 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 29 23:12:13.922462 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.922457 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 29 23:12:13.922569 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.922515 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 29 23:12:13.923213 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.923196 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal"] Oct 29 23:12:13.923323 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.923307 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:13.928602 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.928579 2575 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 29 23:12:13.928739 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.928723 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-dbpfq" Oct 29 23:12:13.931597 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.931577 2575 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 29 23:12:13.931770 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.931738 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal"] Oct 29 23:12:13.938726 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.938708 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dbpfq" Oct 29 23:12:13.977795 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.977777 2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 29 23:12:13.989300 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989274 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlhz5\" (UniqueName: \"kubernetes.io/projected/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-kube-api-access-jlhz5\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:13.989417 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989315 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-socket-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:13.989417 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989334 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r5qh\" (UniqueName: \"kubernetes.io/projected/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-kube-api-access-4r5qh\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:13.989417 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989350 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-sys\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:13.989417 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989377 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:13.989591 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989419 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-cnibin\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:13.989591 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989463 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfhfl\" (UniqueName: \"kubernetes.io/projected/21385963-6be4-4e3f-a996-f48b0c9ab0b1-kube-api-access-jfhfl\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:13.989591 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989512 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4mr2\" (UniqueName: \"kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2\") pod \"network-check-target-dn96d\" (UID: \"b368f5ac-f18a-49d2-b9be-6cca37b7505b\") " pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:13.989591 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989573 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-etc-selinux\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:13.989763 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989626 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-tmp\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:13.989763 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989655 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87lrl\" (UniqueName: \"kubernetes.io/projected/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-kube-api-access-87lrl\") pod \"node-ca-jg249\" (UID: \"c652cee1-24f4-460a-8e1a-669b0ae9c0b1\") " pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:13.989763 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989681 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-etc-kubernetes\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:13.989763 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989706 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-ovnkube-script-lib\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.989763 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989741 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c964bbd5-fa82-459f-b464-d27fd1080654-agent-certs\") pod \"konnectivity-agent-25m4p\" (UID: \"c964bbd5-fa82-459f-b464-d27fd1080654\") " pod="kube-system/konnectivity-agent-25m4p" Oct 29 23:12:13.989960 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989785 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c964bbd5-fa82-459f-b464-d27fd1080654-konnectivity-ca\") pod \"konnectivity-agent-25m4p\" (UID: \"c964bbd5-fa82-459f-b464-d27fd1080654\") " pod="kube-system/konnectivity-agent-25m4p" Oct 29 23:12:13.989960 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989812 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-serviceca\") pod \"node-ca-jg249\" (UID: \"c652cee1-24f4-460a-8e1a-669b0ae9c0b1\") " pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:13.989960 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989839 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-netns\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:13.989960 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989862 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-kubelet\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:13.989960 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989885 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1211641d-1d2c-4de0-b993-5f1e84edc09f-hosts-file\") pod \"node-resolver-zr8dl\" (UID: \"1211641d-1d2c-4de0-b993-5f1e84edc09f\") " pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:13.989960 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989906 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-node-log\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.989960 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989942 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:13.990174 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989967 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:13.990174 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.989992 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-cni-bin\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:13.990174 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990014 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-cni-multus\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:13.990174 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990040 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-ovnkube-config\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.990174 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990073 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-device-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:13.990174 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990096 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf41b924-24b7-4a89-9d11-af8b4246fcde-kubelet-config\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:13.990174 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990118 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-system-cni-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:13.990445 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990196 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-cni-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:13.990445 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990220 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-hostroot\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:13.990445 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990244 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-daemon-config\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:13.990445 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990283 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6eccdb56-305f-4a78-b02b-e3f983030542-host-slash\") pod \"iptables-alerter-c9pz5\" (UID: \"6eccdb56-305f-4a78-b02b-e3f983030542\") " pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:13.990445 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990359 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhszk\" (UniqueName: \"kubernetes.io/projected/a1c203ed-13a2-429b-8a07-34751a25ca06-kube-api-access-xhszk\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.990445 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990414 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:13.990445 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990445 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-run\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:13.990722 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990472 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf5x4\" (UniqueName: \"kubernetes.io/projected/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-kube-api-access-qf5x4\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:13.990722 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990490 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-host\") pod \"node-ca-jg249\" (UID: \"c652cee1-24f4-460a-8e1a-669b0ae9c0b1\") " pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:13.990722 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990526 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a543490c-5c05-4468-a0ad-25ebb475e78e-cni-binary-copy\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:13.990722 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990556 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-kubelet\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.990722 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990585 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-systemd-units\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.990722 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990631 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysconfig\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:13.990722 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990687 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-kubernetes\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:13.991047 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990898 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-systemd\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:13.991047 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990925 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-var-lib-kubelet\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:13.991047 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990948 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-host\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:13.991047 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.990972 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.991047 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991008 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-ovn\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.991047 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991036 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a1c203ed-13a2-429b-8a07-34751a25ca06-ovn-node-metrics-cert\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.991243 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991061 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:13.991243 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991076 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysctl-d\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:13.991243 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991089 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf41b924-24b7-4a89-9d11-af8b4246fcde-dbus\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:13.991243 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991108 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1211641d-1d2c-4de0-b993-5f1e84edc09f-tmp-dir\") pod \"node-resolver-zr8dl\" (UID: \"1211641d-1d2c-4de0-b993-5f1e84edc09f\") " pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:13.991243 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991134 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cnibin\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:13.991243 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991172 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysctl-conf\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:13.991243 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991191 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-lib-modules\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:13.991243 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991227 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-slash\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.991523 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991246 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-log-socket\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.991523 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991284 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-run-ovn-kubernetes\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.991523 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991314 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-cni-bin\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.991523 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991330 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6eccdb56-305f-4a78-b02b-e3f983030542-iptables-alerter-script\") pod \"iptables-alerter-c9pz5\" (UID: \"6eccdb56-305f-4a78-b02b-e3f983030542\") " pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:13.991523 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991344 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbjw5\" (UniqueName: \"kubernetes.io/projected/6eccdb56-305f-4a78-b02b-e3f983030542-kube-api-access-nbjw5\") pod \"iptables-alerter-c9pz5\" (UID: \"6eccdb56-305f-4a78-b02b-e3f983030542\") " pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:13.991523 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991359 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-var-lib-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.991523 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991394 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-system-cni-dir\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:13.991523 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991411 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:13.991523 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991426 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-registration-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:13.991523 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991441 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-sys-fs\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:13.991523 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991454 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-tuned\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:13.991523 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991470 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-etc-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.991523 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991484 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-env-overrides\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.991523 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991498 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-modprobe-d\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:13.991935 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991537 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-socket-dir-parent\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:13.991935 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991553 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.991935 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991569 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-k8s-cni-cncf-io\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:13.991935 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991585 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-multus-certs\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:13.991935 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991602 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-run-netns\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.991935 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991625 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-systemd\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.991935 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991641 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-cni-netd\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:13.991935 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991660 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:13.991935 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991680 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh8rg\" (UniqueName: \"kubernetes.io/projected/1211641d-1d2c-4de0-b993-5f1e84edc09f-kube-api-access-gh8rg\") pod \"node-resolver-zr8dl\" (UID: \"1211641d-1d2c-4de0-b993-5f1e84edc09f\") " pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:13.991935 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991694 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-conf-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:13.991935 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:13.991709 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4748v\" (UniqueName: \"kubernetes.io/projected/a543490c-5c05-4468-a0ad-25ebb475e78e-kube-api-access-4748v\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.011498 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.011559 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.011692 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.011913 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-sw5tb\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.011925 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-zfk9z\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.011983 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.012018 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.012098 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.012143 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.012187 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.012200 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.012230 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h2str\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.012240 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.012275 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.012097 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-5wvkd\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.012299 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-96xtz\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.012145 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.012326 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 29 23:12:14.017222 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.012403 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 29 23:12:14.064387 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.064332 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" event={"ID":"966fb6b19f49ae6dfc8b44441b931f34","Type":"ContainerStarted","Data":"f4267b48ecd99c04495ab3c0a8971ee0931c33620e4b6c0e69d7338f0365f59a"} Oct 29 23:12:14.065857 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.065816 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" event={"ID":"47e5a52938e8a36be364af182a627291","Type":"ContainerStarted","Data":"579027c84fc7add6a266a6a9736f729b2d15b74fc0c1f2f9317ffdd6d25c5d9d"} Oct 29 23:12:14.092061 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.091983 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.092061 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092027 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-ovn\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.092061 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092057 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a1c203ed-13a2-429b-8a07-34751a25ca06-ovn-node-metrics-cert\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.092320 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092083 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:14.092320 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092093 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.092320 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092106 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysctl-d\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.092320 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092129 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf41b924-24b7-4a89-9d11-af8b4246fcde-dbus\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:14.092320 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092151 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1211641d-1d2c-4de0-b993-5f1e84edc09f-tmp-dir\") pod \"node-resolver-zr8dl\" (UID: \"1211641d-1d2c-4de0-b993-5f1e84edc09f\") " pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:14.092320 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092168 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cnibin\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:14.092320 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092183 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysctl-conf\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.092320 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092196 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-lib-modules\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.092320 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092212 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-slash\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.092320 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092234 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-log-socket\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.092320 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092249 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-run-ovn-kubernetes\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.092320 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092279 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-cni-bin\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.092320 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092298 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6eccdb56-305f-4a78-b02b-e3f983030542-iptables-alerter-script\") pod \"iptables-alerter-c9pz5\" (UID: \"6eccdb56-305f-4a78-b02b-e3f983030542\") " pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:14.092320 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092323 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbjw5\" (UniqueName: \"kubernetes.io/projected/6eccdb56-305f-4a78-b02b-e3f983030542-kube-api-access-nbjw5\") pod \"iptables-alerter-c9pz5\" (UID: \"6eccdb56-305f-4a78-b02b-e3f983030542\") " pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:14.092966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092341 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-var-lib-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.092966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092362 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-system-cni-dir\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:14.092966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092378 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:14.092966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092394 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-registration-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:14.092966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092408 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-sys-fs\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:14.092966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092424 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-tuned\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.092966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092445 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-etc-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.092966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092461 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-env-overrides\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.092966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092487 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-modprobe-d\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.092966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092494 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-ovn\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.092966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092503 2575 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 29 23:12:14.092966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092512 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-socket-dir-parent\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.092966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092538 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.092966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092543 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-cni-bin\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.092966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092562 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-k8s-cni-cncf-io\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.092966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092582 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-multus-certs\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.092966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092597 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-run-netns\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.093732 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092612 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-systemd\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.093732 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092628 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-cni-netd\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.093732 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092647 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:14.093732 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092667 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh8rg\" (UniqueName: \"kubernetes.io/projected/1211641d-1d2c-4de0-b993-5f1e84edc09f-kube-api-access-gh8rg\") pod \"node-resolver-zr8dl\" (UID: \"1211641d-1d2c-4de0-b993-5f1e84edc09f\") " pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:14.093732 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092688 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-conf-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.093732 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092718 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4748v\" (UniqueName: \"kubernetes.io/projected/a543490c-5c05-4468-a0ad-25ebb475e78e-kube-api-access-4748v\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.093732 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092748 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlhz5\" (UniqueName: \"kubernetes.io/projected/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-kube-api-access-jlhz5\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:14.093732 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092776 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-socket-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:14.093732 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092792 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r5qh\" (UniqueName: \"kubernetes.io/projected/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-kube-api-access-4r5qh\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:14.093732 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092815 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-sys\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.093732 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092834 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:14.093732 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092848 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-cnibin\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.093732 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092863 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfhfl\" (UniqueName: \"kubernetes.io/projected/21385963-6be4-4e3f-a996-f48b0c9ab0b1-kube-api-access-jfhfl\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:14.093732 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092886 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4mr2\" (UniqueName: \"kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2\") pod \"network-check-target-dn96d\" (UID: \"b368f5ac-f18a-49d2-b9be-6cca37b7505b\") " pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:14.093732 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092913 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-etc-selinux\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:14.093732 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092938 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-tmp\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.093732 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092939 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysctl-conf\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092958 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87lrl\" (UniqueName: \"kubernetes.io/projected/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-kube-api-access-87lrl\") pod \"node-ca-jg249\" (UID: \"c652cee1-24f4-460a-8e1a-669b0ae9c0b1\") " pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092974 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-etc-kubernetes\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.092991 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-ovnkube-script-lib\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093010 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c964bbd5-fa82-459f-b464-d27fd1080654-agent-certs\") pod \"konnectivity-agent-25m4p\" (UID: \"c964bbd5-fa82-459f-b464-d27fd1080654\") " pod="kube-system/konnectivity-agent-25m4p" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093025 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c964bbd5-fa82-459f-b464-d27fd1080654-konnectivity-ca\") pod \"konnectivity-agent-25m4p\" (UID: \"c964bbd5-fa82-459f-b464-d27fd1080654\") " pod="kube-system/konnectivity-agent-25m4p" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093039 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-serviceca\") pod \"node-ca-jg249\" (UID: \"c652cee1-24f4-460a-8e1a-669b0ae9c0b1\") " pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093041 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysctl-d\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093038 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093055 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-netns\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093092 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-netns\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093108 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-kubelet\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093135 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-lib-modules\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093135 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1211641d-1d2c-4de0-b993-5f1e84edc09f-hosts-file\") pod \"node-resolver-zr8dl\" (UID: \"1211641d-1d2c-4de0-b993-5f1e84edc09f\") " pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093172 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-node-log\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093197 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093218 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093244 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-cni-bin\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.094451 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093279 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf41b924-24b7-4a89-9d11-af8b4246fcde-dbus\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:14.094917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093286 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-cni-multus\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.094917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093335 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-cni-multus\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.094917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093355 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-var-lib-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.094917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093377 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-slash\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.094917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093405 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-system-cni-dir\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:14.094917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093412 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-log-socket\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.094917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093448 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-run-ovn-kubernetes\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.094917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093470 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-registration-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:14.094917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093501 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-node-log\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.094917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093525 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1211641d-1d2c-4de0-b993-5f1e84edc09f-tmp-dir\") pod \"node-resolver-zr8dl\" (UID: \"1211641d-1d2c-4de0-b993-5f1e84edc09f\") " pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:14.094917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093575 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-etc-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.094917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093631 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:14.094917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.093970 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-env-overrides\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.094917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094055 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:14.094917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094075 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-modprobe-d\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.094917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094116 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-cni-bin\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.094917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094134 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-socket-dir-parent\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.097316 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094179 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.097316 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094242 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-sys\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.097316 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094318 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-cnibin\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.097316 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094361 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-multus-certs\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.097316 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094367 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-systemd\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.097316 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094399 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-run-netns\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.097316 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094407 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-cni-netd\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.097316 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094451 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:14.097316 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094577 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-conf-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.097316 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094600 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6eccdb56-305f-4a78-b02b-e3f983030542-iptables-alerter-script\") pod \"iptables-alerter-c9pz5\" (UID: \"6eccdb56-305f-4a78-b02b-e3f983030542\") " pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:14.097316 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094662 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cnibin\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:14.097316 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094703 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-kubelet\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.097316 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:14.093529 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 23:12:14.097316 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:14.094798 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs podName:c6fc8782-2d71-4cb3-ab4b-b1d14aad460f nodeName:}" failed. No retries permitted until 2026-10-29 23:12:14.594761346 +0000 UTC m=+3.133484771 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs") pod "network-metrics-daemon-2xktq" (UID: "c6fc8782-2d71-4cb3-ab4b-b1d14aad460f") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 23:12:14.097316 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094832 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-ovnkube-config\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.097316 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094861 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-device-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:14.097316 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094871 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-socket-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094890 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf41b924-24b7-4a89-9d11-af8b4246fcde-kubelet-config\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094913 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-system-cni-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094938 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-cni-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094961 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-hostroot\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094984 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-daemon-config\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095010 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6eccdb56-305f-4a78-b02b-e3f983030542-host-slash\") pod \"iptables-alerter-c9pz5\" (UID: \"6eccdb56-305f-4a78-b02b-e3f983030542\") " pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095016 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-etc-selinux\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095033 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhszk\" (UniqueName: \"kubernetes.io/projected/a1c203ed-13a2-429b-8a07-34751a25ca06-kube-api-access-xhszk\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095057 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095080 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-run\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095103 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf5x4\" (UniqueName: \"kubernetes.io/projected/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-kube-api-access-qf5x4\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095127 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-host\") pod \"node-ca-jg249\" (UID: \"c652cee1-24f4-460a-8e1a-669b0ae9c0b1\") " pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095150 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a543490c-5c05-4468-a0ad-25ebb475e78e-cni-binary-copy\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095173 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-kubelet\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095197 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-systemd-units\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095221 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysconfig\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095246 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-kubernetes\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.098151 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095289 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-systemd\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095314 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-var-lib-kubelet\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095319 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-sys-fs\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095335 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-host\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095421 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-host\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:14.094256 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:14.095478 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret podName:cf41b924-24b7-4a89-9d11-af8b4246fcde nodeName:}" failed. No retries permitted until 2026-10-29 23:12:14.595462923 +0000 UTC m=+3.134186329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret") pod "global-pull-secret-syncer-gz4mg" (UID: "cf41b924-24b7-4a89-9d11-af8b4246fcde") : object "kube-system"/"original-pull-secret" not registered Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095544 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-etc-kubernetes\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095609 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1211641d-1d2c-4de0-b993-5f1e84edc09f-hosts-file\") pod \"node-resolver-zr8dl\" (UID: \"1211641d-1d2c-4de0-b993-5f1e84edc09f\") " pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.094317 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-k8s-cni-cncf-io\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095847 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-device-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095919 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf41b924-24b7-4a89-9d11-af8b4246fcde-kubelet-config\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.095977 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-system-cni-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.096019 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-host\") pod \"node-ca-jg249\" (UID: \"c652cee1-24f4-460a-8e1a-669b0ae9c0b1\") " pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.096033 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-cni-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.096385 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c964bbd5-fa82-459f-b464-d27fd1080654-konnectivity-ca\") pod \"konnectivity-agent-25m4p\" (UID: \"c964bbd5-fa82-459f-b464-d27fd1080654\") " pod="kube-system/konnectivity-agent-25m4p" Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.096457 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6eccdb56-305f-4a78-b02b-e3f983030542-host-slash\") pod \"iptables-alerter-c9pz5\" (UID: \"6eccdb56-305f-4a78-b02b-e3f983030542\") " pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.096510 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-systemd-units\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.098990 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.096552 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-kubelet\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.099831 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.096602 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysconfig\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.099831 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.096665 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-var-lib-kubelet\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.099831 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.096711 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-systemd\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.099831 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.096842 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-ovnkube-script-lib\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.099831 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.096864 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-tuned\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.099831 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.096889 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:14.099831 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.096929 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-hostroot\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.099831 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.096906 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-run\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.099831 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.097032 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a1c203ed-13a2-429b-8a07-34751a25ca06-ovn-node-metrics-cert\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.099831 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.097298 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-serviceca\") pod \"node-ca-jg249\" (UID: \"c652cee1-24f4-460a-8e1a-669b0ae9c0b1\") " pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:14.099831 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.097303 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-ovnkube-config\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.099831 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.097340 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-kubernetes\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.099831 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.097409 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a543490c-5c05-4468-a0ad-25ebb475e78e-cni-binary-copy\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.099831 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.097651 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-daemon-config\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.103231 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.103204 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-tmp\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.107680 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.107657 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c964bbd5-fa82-459f-b464-d27fd1080654-agent-certs\") pod \"konnectivity-agent-25m4p\" (UID: \"c964bbd5-fa82-459f-b464-d27fd1080654\") " pod="kube-system/konnectivity-agent-25m4p" Oct 29 23:12:14.121037 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:14.121016 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 29 23:12:14.121146 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:14.121067 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 29 23:12:14.121146 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:14.121084 2575 projected.go:196] Error preparing data for projected volume kube-api-access-t4mr2 for pod openshift-network-diagnostics/network-check-target-dn96d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 23:12:14.121291 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:14.121154 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2 podName:b368f5ac-f18a-49d2-b9be-6cca37b7505b nodeName:}" failed. No retries permitted until 2026-10-29 23:12:14.621133325 +0000 UTC m=+3.159856748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-t4mr2" (UniqueName: "kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2") pod "network-check-target-dn96d" (UID: "b368f5ac-f18a-49d2-b9be-6cca37b7505b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 23:12:14.123377 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.123276 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfhfl\" (UniqueName: \"kubernetes.io/projected/21385963-6be4-4e3f-a996-f48b0c9ab0b1-kube-api-access-jfhfl\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:14.123377 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.123328 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlhz5\" (UniqueName: \"kubernetes.io/projected/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-kube-api-access-jlhz5\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:14.123544 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.123526 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh8rg\" (UniqueName: \"kubernetes.io/projected/1211641d-1d2c-4de0-b993-5f1e84edc09f-kube-api-access-gh8rg\") pod \"node-resolver-zr8dl\" (UID: \"1211641d-1d2c-4de0-b993-5f1e84edc09f\") " pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:14.124123 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.124102 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r5qh\" (UniqueName: \"kubernetes.io/projected/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-kube-api-access-4r5qh\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:14.128133 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.128083 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4748v\" (UniqueName: \"kubernetes.io/projected/a543490c-5c05-4468-a0ad-25ebb475e78e-kube-api-access-4748v\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.128475 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.128448 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhszk\" (UniqueName: \"kubernetes.io/projected/a1c203ed-13a2-429b-8a07-34751a25ca06-kube-api-access-xhszk\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.128803 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.128774 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf5x4\" (UniqueName: \"kubernetes.io/projected/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-kube-api-access-qf5x4\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.128954 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.128928 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87lrl\" (UniqueName: \"kubernetes.io/projected/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-kube-api-access-87lrl\") pod \"node-ca-jg249\" (UID: \"c652cee1-24f4-460a-8e1a-669b0ae9c0b1\") " pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:14.130764 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.130743 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbjw5\" (UniqueName: \"kubernetes.io/projected/6eccdb56-305f-4a78-b02b-e3f983030542-kube-api-access-nbjw5\") pod \"iptables-alerter-c9pz5\" (UID: \"6eccdb56-305f-4a78-b02b-e3f983030542\") " pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:14.212326 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.212295 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:14.219893 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.219862 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:14.223671 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:14.223643 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02e4b65a_b1c0_4ed0_9e62_2a9a428f2f5a.slice/crio-1c8ab03ed880a449f1f58318234ce4880b7421e67cac60de9783d3a0d4f87086 WatchSource:0}: Error finding container 1c8ab03ed880a449f1f58318234ce4880b7421e67cac60de9783d3a0d4f87086: Status 404 returned error can't find the container with id 1c8ab03ed880a449f1f58318234ce4880b7421e67cac60de9783d3a0d4f87086 Oct 29 23:12:14.229951 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.229929 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:14.230975 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:14.230940 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1211641d_1d2c_4de0_b993_5f1e84edc09f.slice/crio-ee7d783cd1ab14cd5213846bce28f544afd16f712d3405efe87a229a08e0e6c3 WatchSource:0}: Error finding container ee7d783cd1ab14cd5213846bce28f544afd16f712d3405efe87a229a08e0e6c3: Status 404 returned error can't find the container with id ee7d783cd1ab14cd5213846bce28f544afd16f712d3405efe87a229a08e0e6c3 Oct 29 23:12:14.256001 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.255977 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-49s6q" Oct 29 23:12:14.262760 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.262735 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:14.266403 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:14.266375 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda543490c_5c05_4468_a0ad_25ebb475e78e.slice/crio-4a780d6c876c96b2ac229f39a9eb1bc78c74918bd006b1fb2016e24048655c15 WatchSource:0}: Error finding container 4a780d6c876c96b2ac229f39a9eb1bc78c74918bd006b1fb2016e24048655c15: Status 404 returned error can't find the container with id 4a780d6c876c96b2ac229f39a9eb1bc78c74918bd006b1fb2016e24048655c15 Oct 29 23:12:14.271862 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.271430 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:14.271862 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:14.271788 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1c203ed_13a2_429b_8a07_34751a25ca06.slice/crio-a8b4722e4e6b2bb95dc0ce5220d860476ec452f37aa982d55f570cc0d55a8c91 WatchSource:0}: Error finding container a8b4722e4e6b2bb95dc0ce5220d860476ec452f37aa982d55f570cc0d55a8c91: Status 404 returned error can't find the container with id a8b4722e4e6b2bb95dc0ce5220d860476ec452f37aa982d55f570cc0d55a8c91 Oct 29 23:12:14.273619 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.273591 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-25m4p" Oct 29 23:12:14.280839 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.280819 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:14.282128 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:14.282104 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eccdb56_305f_4a78_b02b_e3f983030542.slice/crio-50b667f42d34af62fd160fc85feff19064c5a64317e6ce286a2a267e8bf72c07 WatchSource:0}: Error finding container 50b667f42d34af62fd160fc85feff19064c5a64317e6ce286a2a267e8bf72c07: Status 404 returned error can't find the container with id 50b667f42d34af62fd160fc85feff19064c5a64317e6ce286a2a267e8bf72c07 Oct 29 23:12:14.287659 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:14.284883 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc964bbd5_fa82_459f_b464_d27fd1080654.slice/crio-d94ef0ff71197b3c3c5133f0406808e8cc230c141502ffeca63e12e7de44b305 WatchSource:0}: Error finding container d94ef0ff71197b3c3c5133f0406808e8cc230c141502ffeca63e12e7de44b305: Status 404 returned error can't find the container with id d94ef0ff71197b3c3c5133f0406808e8cc230c141502ffeca63e12e7de44b305 Oct 29 23:12:14.287659 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.287496 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:14.292190 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:14.292111 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21385963_6be4_4e3f_a996_f48b0c9ab0b1.slice/crio-93eba272389667f4872d3f6c71e1d9639fc8144342dfee0a04b23f2d93508d3c WatchSource:0}: Error finding container 93eba272389667f4872d3f6c71e1d9639fc8144342dfee0a04b23f2d93508d3c: Status 404 returned error can't find the container with id 93eba272389667f4872d3f6c71e1d9639fc8144342dfee0a04b23f2d93508d3c Oct 29 23:12:14.298346 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:14.298324 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ffa4cb8_b390_49d2_839d_3cf43397eb6f.slice/crio-e2e3cb8499a07a3b027a62e49940d2b429c696b285967e574f469eecea1d3bb2 WatchSource:0}: Error finding container e2e3cb8499a07a3b027a62e49940d2b429c696b285967e574f469eecea1d3bb2: Status 404 returned error can't find the container with id e2e3cb8499a07a3b027a62e49940d2b429c696b285967e574f469eecea1d3bb2 Oct 29 23:12:14.406851 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.406821 2575 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 23:12:14.599208 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.599172 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:14.599434 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.599255 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:14.599434 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:14.599346 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 29 23:12:14.599434 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:14.599351 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 23:12:14.599434 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:14.599404 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs podName:c6fc8782-2d71-4cb3-ab4b-b1d14aad460f nodeName:}" failed. No retries permitted until 2026-10-29 23:12:15.599385706 +0000 UTC m=+4.138109112 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs") pod "network-metrics-daemon-2xktq" (UID: "c6fc8782-2d71-4cb3-ab4b-b1d14aad460f") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 23:12:14.599434 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:14.599421 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret podName:cf41b924-24b7-4a89-9d11-af8b4246fcde nodeName:}" failed. No retries permitted until 2026-10-29 23:12:15.59941198 +0000 UTC m=+4.138135386 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret") pod "global-pull-secret-syncer-gz4mg" (UID: "cf41b924-24b7-4a89-9d11-af8b4246fcde") : object "kube-system"/"original-pull-secret" not registered Oct 29 23:12:14.700516 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.700233 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4mr2\" (UniqueName: \"kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2\") pod \"network-check-target-dn96d\" (UID: \"b368f5ac-f18a-49d2-b9be-6cca37b7505b\") " pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:14.700516 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:14.700490 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 29 23:12:14.700516 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:14.700512 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 29 23:12:14.700707 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:14.700525 2575 projected.go:196] Error preparing data for projected volume kube-api-access-t4mr2 for pod openshift-network-diagnostics/network-check-target-dn96d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 23:12:14.700707 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:14.700585 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2 podName:b368f5ac-f18a-49d2-b9be-6cca37b7505b nodeName:}" failed. No retries permitted until 2026-10-29 23:12:15.700565898 +0000 UTC m=+4.239289305 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-t4mr2" (UniqueName: "kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2") pod "network-check-target-dn96d" (UID: "b368f5ac-f18a-49d2-b9be-6cca37b7505b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 23:12:14.942224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.942034 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-28 23:07:13 +0000 UTC" deadline="2028-04-28 23:10:14.798564372 +0000 UTC" Oct 29 23:12:14.942224 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:14.942097 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13127h57m59.856472325s" Oct 29 23:12:15.075439 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.075326 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-25m4p" event={"ID":"c964bbd5-fa82-459f-b464-d27fd1080654","Type":"ContainerStarted","Data":"d94ef0ff71197b3c3c5133f0406808e8cc230c141502ffeca63e12e7de44b305"} Oct 29 23:12:15.081314 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.081169 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" event={"ID":"a1c203ed-13a2-429b-8a07-34751a25ca06","Type":"ContainerStarted","Data":"a8b4722e4e6b2bb95dc0ce5220d860476ec452f37aa982d55f570cc0d55a8c91"} Oct 29 23:12:15.085365 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.085286 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jg249" event={"ID":"c652cee1-24f4-460a-8e1a-669b0ae9c0b1","Type":"ContainerStarted","Data":"91e59f285c148ed7601fd08e73b9fdd80c84a9a2e14036b88020ad258d7218ea"} Oct 29 23:12:15.091098 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.090990 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zr8dl" event={"ID":"1211641d-1d2c-4de0-b993-5f1e84edc09f","Type":"ContainerStarted","Data":"ee7d783cd1ab14cd5213846bce28f544afd16f712d3405efe87a229a08e0e6c3"} Oct 29 23:12:15.115985 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.114972 2575 generic.go:356] "Generic (PLEG): container finished" podID="966fb6b19f49ae6dfc8b44441b931f34" containerID="9db485ec3f27847f847194b0e2f47c9c84d0de41a34532233cc52f1e059b4ef5" exitCode=0 Oct 29 23:12:15.115985 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.115056 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" event={"ID":"966fb6b19f49ae6dfc8b44441b931f34","Type":"ContainerDied","Data":"9db485ec3f27847f847194b0e2f47c9c84d0de41a34532233cc52f1e059b4ef5"} Oct 29 23:12:15.130193 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.130100 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" event={"ID":"7ffa4cb8-b390-49d2-839d-3cf43397eb6f","Type":"ContainerStarted","Data":"e2e3cb8499a07a3b027a62e49940d2b429c696b285967e574f469eecea1d3bb2"} Oct 29 23:12:15.138436 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.138368 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c9pz5" event={"ID":"6eccdb56-305f-4a78-b02b-e3f983030542","Type":"ContainerStarted","Data":"50b667f42d34af62fd160fc85feff19064c5a64317e6ce286a2a267e8bf72c07"} Oct 29 23:12:15.140785 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.140763 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-49s6q" event={"ID":"a543490c-5c05-4468-a0ad-25ebb475e78e","Type":"ContainerStarted","Data":"4a780d6c876c96b2ac229f39a9eb1bc78c74918bd006b1fb2016e24048655c15"} Oct 29 23:12:15.143074 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.143002 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" event={"ID":"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a","Type":"ContainerStarted","Data":"1c8ab03ed880a449f1f58318234ce4880b7421e67cac60de9783d3a0d4f87086"} Oct 29 23:12:15.156917 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.156540 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" event={"ID":"47e5a52938e8a36be364af182a627291","Type":"ContainerStarted","Data":"a9861ee4714c733778bb4d729d3ef68bcf0ec1234ac4b2437cd43d75100546e6"} Oct 29 23:12:15.161028 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.160961 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzb6b" event={"ID":"21385963-6be4-4e3f-a996-f48b0c9ab0b1","Type":"ContainerStarted","Data":"93eba272389667f4872d3f6c71e1d9639fc8144342dfee0a04b23f2d93508d3c"} Oct 29 23:12:15.199002 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.198911 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" podStartSLOduration=2.198892221 podStartE2EDuration="2.198892221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:15.198712541 +0000 UTC m=+3.737435970" watchObservedRunningTime="2026-10-29 23:12:15.198892221 +0000 UTC m=+3.737615650" Oct 29 23:12:15.607320 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.607222 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:15.607320 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.607304 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:15.607530 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:15.607464 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 29 23:12:15.607530 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:15.607527 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret podName:cf41b924-24b7-4a89-9d11-af8b4246fcde nodeName:}" failed. No retries permitted until 2026-10-29 23:12:17.607508401 +0000 UTC m=+6.146231827 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret") pod "global-pull-secret-syncer-gz4mg" (UID: "cf41b924-24b7-4a89-9d11-af8b4246fcde") : object "kube-system"/"original-pull-secret" not registered Oct 29 23:12:15.607934 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:15.607917 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 23:12:15.608022 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:15.607972 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs podName:c6fc8782-2d71-4cb3-ab4b-b1d14aad460f nodeName:}" failed. No retries permitted until 2026-10-29 23:12:17.607956984 +0000 UTC m=+6.146680406 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs") pod "network-metrics-daemon-2xktq" (UID: "c6fc8782-2d71-4cb3-ab4b-b1d14aad460f") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 23:12:15.707914 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.707883 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4mr2\" (UniqueName: \"kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2\") pod \"network-check-target-dn96d\" (UID: \"b368f5ac-f18a-49d2-b9be-6cca37b7505b\") " pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:15.708076 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:15.708034 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 29 23:12:15.708076 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:15.708052 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 29 23:12:15.708076 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:15.708066 2575 projected.go:196] Error preparing data for projected volume kube-api-access-t4mr2 for pod openshift-network-diagnostics/network-check-target-dn96d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 23:12:15.708241 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:15.708122 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2 podName:b368f5ac-f18a-49d2-b9be-6cca37b7505b nodeName:}" failed. No retries permitted until 2026-10-29 23:12:17.708110212 +0000 UTC m=+6.246833630 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-t4mr2" (UniqueName: "kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2") pod "network-check-target-dn96d" (UID: "b368f5ac-f18a-49d2-b9be-6cca37b7505b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 23:12:15.942711 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.942612 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-28 23:07:13 +0000 UTC" deadline="2028-05-30 19:20:56.60961765 +0000 UTC" Oct 29 23:12:15.942711 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:15.942654 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13892h8m40.666967477s" Oct 29 23:12:16.063547 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:16.062791 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:16.063547 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:16.062926 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dn96d" podUID="b368f5ac-f18a-49d2-b9be-6cca37b7505b" Oct 29 23:12:16.063547 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:16.063436 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:16.063806 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:16.063565 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gz4mg" podUID="cf41b924-24b7-4a89-9d11-af8b4246fcde" Oct 29 23:12:16.063806 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:16.063595 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:16.063806 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:16.063670 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xktq" podUID="c6fc8782-2d71-4cb3-ab4b-b1d14aad460f" Oct 29 23:12:16.199347 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:16.197737 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" event={"ID":"966fb6b19f49ae6dfc8b44441b931f34","Type":"ContainerStarted","Data":"8dc18baec274648967ab86ab667acdcaafc3bda7b6b480bf2e49fe9fa5623414"} Oct 29 23:12:16.247094 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:16.247029 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" podStartSLOduration=3.247009348 podStartE2EDuration="3.247009348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:16.245515922 +0000 UTC m=+4.784239351" watchObservedRunningTime="2026-10-29 23:12:16.247009348 +0000 UTC m=+4.785732778" Oct 29 23:12:17.206989 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:17.206873 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzb6b" event={"ID":"21385963-6be4-4e3f-a996-f48b0c9ab0b1","Type":"ContainerStarted","Data":"b4f82737db306c147ad7dfa3c215526641372f4f8708bb52a43238dfce60f50a"} Oct 29 23:12:17.625411 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:17.625367 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:17.625596 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:17.625428 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:17.625596 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:17.625569 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 29 23:12:17.625707 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:17.625672 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret podName:cf41b924-24b7-4a89-9d11-af8b4246fcde nodeName:}" failed. No retries permitted until 2026-10-29 23:12:21.625631014 +0000 UTC m=+10.164354425 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret") pod "global-pull-secret-syncer-gz4mg" (UID: "cf41b924-24b7-4a89-9d11-af8b4246fcde") : object "kube-system"/"original-pull-secret" not registered Oct 29 23:12:17.626213 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:17.626118 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 23:12:17.626213 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:17.626176 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs podName:c6fc8782-2d71-4cb3-ab4b-b1d14aad460f nodeName:}" failed. No retries permitted until 2026-10-29 23:12:21.626159754 +0000 UTC m=+10.164883164 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs") pod "network-metrics-daemon-2xktq" (UID: "c6fc8782-2d71-4cb3-ab4b-b1d14aad460f") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 23:12:17.727007 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:17.726438 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4mr2\" (UniqueName: \"kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2\") pod \"network-check-target-dn96d\" (UID: \"b368f5ac-f18a-49d2-b9be-6cca37b7505b\") " pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:17.727007 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:17.726613 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 29 23:12:17.727007 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:17.726629 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 29 23:12:17.727007 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:17.726638 2575 projected.go:196] Error preparing data for projected volume kube-api-access-t4mr2 for pod openshift-network-diagnostics/network-check-target-dn96d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 23:12:17.727007 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:17.726743 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2 podName:b368f5ac-f18a-49d2-b9be-6cca37b7505b nodeName:}" failed. No retries permitted until 2026-10-29 23:12:21.72666897 +0000 UTC m=+10.265392376 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-t4mr2" (UniqueName: "kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2") pod "network-check-target-dn96d" (UID: "b368f5ac-f18a-49d2-b9be-6cca37b7505b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 23:12:18.062619 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:18.061865 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:18.062619 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:18.062005 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gz4mg" podUID="cf41b924-24b7-4a89-9d11-af8b4246fcde" Oct 29 23:12:18.062619 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:18.062391 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:18.062619 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:18.062475 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dn96d" podUID="b368f5ac-f18a-49d2-b9be-6cca37b7505b" Oct 29 23:12:18.063172 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:18.062994 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:18.063172 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:18.063118 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xktq" podUID="c6fc8782-2d71-4cb3-ab4b-b1d14aad460f" Oct 29 23:12:18.212946 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:18.212910 2575 generic.go:356] "Generic (PLEG): container finished" podID="21385963-6be4-4e3f-a996-f48b0c9ab0b1" containerID="b4f82737db306c147ad7dfa3c215526641372f4f8708bb52a43238dfce60f50a" exitCode=0 Oct 29 23:12:18.213531 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:18.212972 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzb6b" event={"ID":"21385963-6be4-4e3f-a996-f48b0c9ab0b1","Type":"ContainerDied","Data":"b4f82737db306c147ad7dfa3c215526641372f4f8708bb52a43238dfce60f50a"} Oct 29 23:12:19.226817 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:19.226611 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-25m4p" event={"ID":"c964bbd5-fa82-459f-b464-d27fd1080654","Type":"ContainerStarted","Data":"43173893d47e69ad85991ba2ef4860a695fa59785aea5db44f1e3c73108e29da"} Oct 29 23:12:20.061479 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:20.061407 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:20.061606 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:20.061526 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dn96d" podUID="b368f5ac-f18a-49d2-b9be-6cca37b7505b" Oct 29 23:12:20.061606 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:20.061571 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:20.061666 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:20.061654 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xktq" podUID="c6fc8782-2d71-4cb3-ab4b-b1d14aad460f" Oct 29 23:12:20.061815 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:20.061702 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:20.061815 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:20.061767 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gz4mg" podUID="cf41b924-24b7-4a89-9d11-af8b4246fcde" Oct 29 23:12:20.233571 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:20.233534 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" event={"ID":"7ffa4cb8-b390-49d2-839d-3cf43397eb6f","Type":"ContainerStarted","Data":"5d1ec2565dc50014b6744039b9d895c25bbc03187db040a1f22d0fa35d1f660e"} Oct 29 23:12:20.237242 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:20.237215 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jg249" event={"ID":"c652cee1-24f4-460a-8e1a-669b0ae9c0b1","Type":"ContainerStarted","Data":"c6b40a44566508817c1fe6a7f1106d43edc005d489df762b943962e4320141f5"} Oct 29 23:12:20.278532 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:20.278476 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-25m4p" podStartSLOduration=4.46780742 podStartE2EDuration="8.278457585s" totalImagesPullingTime="3.810650165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:19.27202328 +0000 UTC m=+7.810746708" watchObservedRunningTime="2026-10-29 23:12:20.278457585 +0000 UTC m=+8.817181014" Oct 29 23:12:21.449863 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:21.449803 2575 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 29 23:12:21.658370 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:21.658287 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:21.658501 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:21.658377 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:21.658658 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:21.658633 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 29 23:12:21.658748 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:21.658718 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret podName:cf41b924-24b7-4a89-9d11-af8b4246fcde nodeName:}" failed. No retries permitted until 2026-10-29 23:12:29.658697924 +0000 UTC m=+18.197421353 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret") pod "global-pull-secret-syncer-gz4mg" (UID: "cf41b924-24b7-4a89-9d11-af8b4246fcde") : object "kube-system"/"original-pull-secret" not registered Oct 29 23:12:21.659125 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:21.659105 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 23:12:21.659213 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:21.659161 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs podName:c6fc8782-2d71-4cb3-ab4b-b1d14aad460f nodeName:}" failed. No retries permitted until 2026-10-29 23:12:29.659147973 +0000 UTC m=+18.197871395 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs") pod "network-metrics-daemon-2xktq" (UID: "c6fc8782-2d71-4cb3-ab4b-b1d14aad460f") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 23:12:21.759099 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:21.759063 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4mr2\" (UniqueName: \"kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2\") pod \"network-check-target-dn96d\" (UID: \"b368f5ac-f18a-49d2-b9be-6cca37b7505b\") " pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:21.759240 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:21.759223 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 29 23:12:21.759240 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:21.759237 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 29 23:12:21.759376 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:21.759248 2575 projected.go:196] Error preparing data for projected volume kube-api-access-t4mr2 for pod openshift-network-diagnostics/network-check-target-dn96d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 23:12:21.759376 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:21.759312 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2 podName:b368f5ac-f18a-49d2-b9be-6cca37b7505b nodeName:}" failed. No retries permitted until 2026-10-29 23:12:29.759295953 +0000 UTC m=+18.298019374 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-t4mr2" (UniqueName: "kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2") pod "network-check-target-dn96d" (UID: "b368f5ac-f18a-49d2-b9be-6cca37b7505b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 23:12:21.961474 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:21.961304 2575 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-29T23:12:21.4498244Z","UUID":"7540d65a-975c-4921-b91e-c2e6f6186655","Handler":null,"Name":"","Endpoint":""} Oct 29 23:12:21.961839 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:21.961809 2575 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-25m4p" Oct 29 23:12:21.967530 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:21.967502 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 29 23:12:21.967530 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:21.967538 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 29 23:12:21.968738 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:21.968209 2575 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-25m4p" Oct 29 23:12:22.012181 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:22.010780 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jg249" podStartSLOduration=5.072382654 podStartE2EDuration="10.010760692s" totalImagesPullingTime="4.938378038s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:20.278742636 +0000 UTC m=+8.817466065" watchObservedRunningTime="2026-10-29 23:12:22.010760692 +0000 UTC m=+10.549484124" Oct 29 23:12:22.063441 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:22.063401 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:22.063441 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:22.063438 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:22.063647 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:22.063564 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gz4mg" podUID="cf41b924-24b7-4a89-9d11-af8b4246fcde" Oct 29 23:12:22.064547 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:22.063792 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:22.064547 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:22.063918 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xktq" podUID="c6fc8782-2d71-4cb3-ab4b-b1d14aad460f" Oct 29 23:12:22.064547 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:22.063983 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dn96d" podUID="b368f5ac-f18a-49d2-b9be-6cca37b7505b" Oct 29 23:12:22.243725 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:22.243642 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zr8dl" event={"ID":"1211641d-1d2c-4de0-b993-5f1e84edc09f","Type":"ContainerStarted","Data":"3cffbd0fb8a66f6aef871f6faacec98ecd0e82e49e960f440275ba2272d5da48"} Oct 29 23:12:22.246726 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:22.246696 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" event={"ID":"7ffa4cb8-b390-49d2-839d-3cf43397eb6f","Type":"ContainerStarted","Data":"ebd8458efcbc7401b5e845a0b42f1c4a734659ce26b1898f1216c27ee5443f6f"} Oct 29 23:12:22.249276 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:22.249237 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-49s6q" event={"ID":"a543490c-5c05-4468-a0ad-25ebb475e78e","Type":"ContainerStarted","Data":"c2584fb48b7700805d187cba075f4b64a9c5a24eb8a9b2c3f260883d5e0ce9cb"} Oct 29 23:12:22.287067 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:22.287007 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-zr8dl" podStartSLOduration=2.627435002 podStartE2EDuration="10.286990199s" totalImagesPullingTime="7.659555197s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:22.286326946 +0000 UTC m=+10.825050376" watchObservedRunningTime="2026-10-29 23:12:22.286990199 +0000 UTC m=+10.825713628" Oct 29 23:12:22.331293 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:22.331225 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-49s6q" podStartSLOduration=2.795537913 podStartE2EDuration="10.331212856s" totalImagesPullingTime="7.535674943s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:22.330535914 +0000 UTC m=+10.869259342" watchObservedRunningTime="2026-10-29 23:12:22.331212856 +0000 UTC m=+10.869936284" Oct 29 23:12:23.253037 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:23.252788 2575 generic.go:356] "Generic (PLEG): container finished" podID="21385963-6be4-4e3f-a996-f48b0c9ab0b1" containerID="72d5bae21d7690a330bd75bfa6a185e5373724ca00c190e06d185b07fd3e8334" exitCode=0 Oct 29 23:12:23.253528 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:23.252942 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzb6b" event={"ID":"21385963-6be4-4e3f-a996-f48b0c9ab0b1","Type":"ContainerDied","Data":"72d5bae21d7690a330bd75bfa6a185e5373724ca00c190e06d185b07fd3e8334"} Oct 29 23:12:23.256818 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:23.256796 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" event={"ID":"7ffa4cb8-b390-49d2-839d-3cf43397eb6f","Type":"ContainerStarted","Data":"f7fb9ecbcbdd9e59e0aa21ff81b7c8ed941101d9343e37bbda4f0f12bc1cfcbd"} Oct 29 23:12:23.258860 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:23.258838 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c9pz5" event={"ID":"6eccdb56-305f-4a78-b02b-e3f983030542","Type":"ContainerStarted","Data":"4c498fb5b999999afc0b2d2723697b0ec7249ec752638c8f8219fb0751e518b6"} Oct 29 23:12:23.355745 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:23.355690 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-c9pz5" podStartSLOduration=3.929263316 podStartE2EDuration="11.355676102s" totalImagesPullingTime="7.426412786s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:23.355445151 +0000 UTC m=+11.894168578" watchObservedRunningTime="2026-10-29 23:12:23.355676102 +0000 UTC m=+11.894399529" Oct 29 23:12:24.060434 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:24.060363 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:24.060434 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:24.060376 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:24.060631 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:24.060477 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xktq" podUID="c6fc8782-2d71-4cb3-ab4b-b1d14aad460f" Oct 29 23:12:24.060631 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:24.060559 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dn96d" podUID="b368f5ac-f18a-49d2-b9be-6cca37b7505b" Oct 29 23:12:24.060631 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:24.060601 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:24.060783 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:24.060680 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gz4mg" podUID="cf41b924-24b7-4a89-9d11-af8b4246fcde" Oct 29 23:12:24.262296 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:24.262243 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" event={"ID":"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a","Type":"ContainerStarted","Data":"c6bc65258fb763bdb493cd6c29c413f6574b047e1a5e124aa29a9f012fd21a76"} Oct 29 23:12:24.264423 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:24.264395 2575 generic.go:356] "Generic (PLEG): container finished" podID="21385963-6be4-4e3f-a996-f48b0c9ab0b1" containerID="6380f9bdf6abcbb9d9ce03a1588d92b96a0d504bbf26b72af2eddeae75d854a0" exitCode=0 Oct 29 23:12:24.264553 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:24.264469 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzb6b" event={"ID":"21385963-6be4-4e3f-a996-f48b0c9ab0b1","Type":"ContainerDied","Data":"6380f9bdf6abcbb9d9ce03a1588d92b96a0d504bbf26b72af2eddeae75d854a0"} Oct 29 23:12:24.306107 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:24.306053 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" podStartSLOduration=4.390071946 podStartE2EDuration="12.306038094s" totalImagesPullingTime="7.915966148s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:23.399981168 +0000 UTC m=+11.938704597" watchObservedRunningTime="2026-10-29 23:12:24.306038094 +0000 UTC m=+12.844761522" Oct 29 23:12:24.306242 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:24.306194 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" podStartSLOduration=2.635997095 podStartE2EDuration="12.306184812s" totalImagesPullingTime="9.670187717s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:24.306155232 +0000 UTC m=+12.844878662" watchObservedRunningTime="2026-10-29 23:12:24.306184812 +0000 UTC m=+12.844908240" Oct 29 23:12:26.059570 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:26.059348 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:26.060053 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:26.059668 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xktq" podUID="c6fc8782-2d71-4cb3-ab4b-b1d14aad460f" Oct 29 23:12:26.060314 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:26.060292 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:26.060427 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:26.060370 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gz4mg" podUID="cf41b924-24b7-4a89-9d11-af8b4246fcde" Oct 29 23:12:26.060478 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:26.060435 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:26.060520 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:26.060505 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dn96d" podUID="b368f5ac-f18a-49d2-b9be-6cca37b7505b" Oct 29 23:12:26.270773 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:26.270740 2575 generic.go:356] "Generic (PLEG): container finished" podID="21385963-6be4-4e3f-a996-f48b0c9ab0b1" containerID="1f9da28be6977a4561a105d6bd2ba8b40fa61f5d9d0517443fb01373f8adb128" exitCode=0 Oct 29 23:12:26.270926 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:26.270805 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzb6b" event={"ID":"21385963-6be4-4e3f-a996-f48b0c9ab0b1","Type":"ContainerDied","Data":"1f9da28be6977a4561a105d6bd2ba8b40fa61f5d9d0517443fb01373f8adb128"} Oct 29 23:12:28.062782 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:28.062756 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:28.063122 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:28.062866 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:28.063122 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:28.062878 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:28.063122 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:28.062878 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gz4mg" podUID="cf41b924-24b7-4a89-9d11-af8b4246fcde" Oct 29 23:12:28.063122 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:28.062931 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dn96d" podUID="b368f5ac-f18a-49d2-b9be-6cca37b7505b" Oct 29 23:12:28.063122 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:28.063016 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xktq" podUID="c6fc8782-2d71-4cb3-ab4b-b1d14aad460f" Oct 29 23:12:29.280602 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:29.280373 2575 generic.go:356] "Generic (PLEG): container finished" podID="21385963-6be4-4e3f-a996-f48b0c9ab0b1" containerID="b143e3f545d53f57349cdf10ea2361787b9b8e7bb85cca1764a20730f93d5219" exitCode=0 Oct 29 23:12:29.280602 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:29.280420 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzb6b" event={"ID":"21385963-6be4-4e3f-a996-f48b0c9ab0b1","Type":"ContainerDied","Data":"b143e3f545d53f57349cdf10ea2361787b9b8e7bb85cca1764a20730f93d5219"} Oct 29 23:12:29.283534 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:29.283511 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" event={"ID":"a1c203ed-13a2-429b-8a07-34751a25ca06","Type":"ContainerStarted","Data":"9951ca4fc67f6ed390846f1bf3036372ceb325a5fb4f3c1266852e605fc82223"} Oct 29 23:12:29.283663 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:29.283585 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" event={"ID":"a1c203ed-13a2-429b-8a07-34751a25ca06","Type":"ContainerStarted","Data":"e13f59c06b5b931672f855bd17dbcbb911432fd13b515900f7e866491e247837"} Oct 29 23:12:29.283663 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:29.283633 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" event={"ID":"a1c203ed-13a2-429b-8a07-34751a25ca06","Type":"ContainerStarted","Data":"bbec61fc4cc9d8c62b4b9c79aa6b34d715845ab7b268857989dcc57c92abe902"} Oct 29 23:12:29.283774 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:29.283676 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" event={"ID":"a1c203ed-13a2-429b-8a07-34751a25ca06","Type":"ContainerStarted","Data":"1d8b5a7194da52b361b5748623ab8c04958b940a3fa8df2badb089be8a4f842a"} Oct 29 23:12:29.283774 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:29.283723 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" event={"ID":"a1c203ed-13a2-429b-8a07-34751a25ca06","Type":"ContainerStarted","Data":"6391168b1ed40240f95619c8071bf91625a9fcd4ec54c97fa28e148a3e73704d"} Oct 29 23:12:29.283906 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:29.283775 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" event={"ID":"a1c203ed-13a2-429b-8a07-34751a25ca06","Type":"ContainerStarted","Data":"2c6fa48a6106bd8ca83c0680ee8873811de422b7b7ad0e9a93cd0b2f2eb4dcf8"} Oct 29 23:12:29.715689 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:29.715663 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:29.715842 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:29.715701 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:29.715842 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:29.715794 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 29 23:12:29.715842 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:29.715795 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 23:12:29.715842 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:29.715837 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret podName:cf41b924-24b7-4a89-9d11-af8b4246fcde nodeName:}" failed. No retries permitted until 2026-10-29 23:12:45.715824854 +0000 UTC m=+34.254548260 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret") pod "global-pull-secret-syncer-gz4mg" (UID: "cf41b924-24b7-4a89-9d11-af8b4246fcde") : object "kube-system"/"original-pull-secret" not registered Oct 29 23:12:29.716007 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:29.715850 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs podName:c6fc8782-2d71-4cb3-ab4b-b1d14aad460f nodeName:}" failed. No retries permitted until 2026-10-29 23:12:45.715843805 +0000 UTC m=+34.254567211 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs") pod "network-metrics-daemon-2xktq" (UID: "c6fc8782-2d71-4cb3-ab4b-b1d14aad460f") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 29 23:12:29.816462 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:29.816436 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4mr2\" (UniqueName: \"kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2\") pod \"network-check-target-dn96d\" (UID: \"b368f5ac-f18a-49d2-b9be-6cca37b7505b\") " pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:29.816628 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:29.816536 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 29 23:12:29.816628 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:29.816548 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 29 23:12:29.816628 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:29.816557 2575 projected.go:196] Error preparing data for projected volume kube-api-access-t4mr2 for pod openshift-network-diagnostics/network-check-target-dn96d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 23:12:29.816628 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:29.816598 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2 podName:b368f5ac-f18a-49d2-b9be-6cca37b7505b nodeName:}" failed. No retries permitted until 2026-10-29 23:12:45.816585495 +0000 UTC m=+34.355308901 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-t4mr2" (UniqueName: "kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2") pod "network-check-target-dn96d" (UID: "b368f5ac-f18a-49d2-b9be-6cca37b7505b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 29 23:12:30.059659 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:30.059580 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:30.059659 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:30.059598 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:30.059659 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:30.059610 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:30.059921 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:30.059702 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gz4mg" podUID="cf41b924-24b7-4a89-9d11-af8b4246fcde" Oct 29 23:12:30.059921 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:30.059728 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dn96d" podUID="b368f5ac-f18a-49d2-b9be-6cca37b7505b" Oct 29 23:12:30.059921 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:30.059798 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xktq" podUID="c6fc8782-2d71-4cb3-ab4b-b1d14aad460f" Oct 29 23:12:30.288740 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:30.288706 2575 generic.go:356] "Generic (PLEG): container finished" podID="21385963-6be4-4e3f-a996-f48b0c9ab0b1" containerID="f494c37fbb7a64c0269ebee03482b5b95d888f0c60866d8cbfaff9fd48b872e2" exitCode=0 Oct 29 23:12:30.289353 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:30.288766 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzb6b" event={"ID":"21385963-6be4-4e3f-a996-f48b0c9ab0b1","Type":"ContainerDied","Data":"f494c37fbb7a64c0269ebee03482b5b95d888f0c60866d8cbfaff9fd48b872e2"} Oct 29 23:12:31.293913 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:31.293830 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzb6b" event={"ID":"21385963-6be4-4e3f-a996-f48b0c9ab0b1","Type":"ContainerStarted","Data":"14f0fbc9e9458f749bc97b1145de61f5e9ad349c5456a83614da7eb024736f2b"} Oct 29 23:12:31.296856 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:31.296832 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" event={"ID":"a1c203ed-13a2-429b-8a07-34751a25ca06","Type":"ContainerStarted","Data":"22f789c3305b362d43f173bbb396f81518e29c1f23747b64120dc55a3b1023d7"} Oct 29 23:12:31.354962 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:31.354913 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xzb6b" podStartSLOduration=8.841581111 podStartE2EDuration="19.354897263s" totalImagesPullingTime="10.513316152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:31.354778938 +0000 UTC m=+19.893502365" watchObservedRunningTime="2026-10-29 23:12:31.354897263 +0000 UTC m=+19.893620693" Oct 29 23:12:32.060179 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:32.060147 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:32.060344 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:32.060296 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dn96d" podUID="b368f5ac-f18a-49d2-b9be-6cca37b7505b" Oct 29 23:12:32.060344 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:32.060320 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:32.060443 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:32.060398 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xktq" podUID="c6fc8782-2d71-4cb3-ab4b-b1d14aad460f" Oct 29 23:12:32.060443 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:32.060434 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:32.060503 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:32.060489 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gz4mg" podUID="cf41b924-24b7-4a89-9d11-af8b4246fcde" Oct 29 23:12:33.305363 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:33.305147 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" event={"ID":"a1c203ed-13a2-429b-8a07-34751a25ca06","Type":"ContainerStarted","Data":"ffcb45a209f21c3e190dde27ba40f7367ac351c7715d56d1f1ba765df24379d6"} Oct 29 23:12:33.306061 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:33.305504 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:33.306061 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:33.305565 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:33.320940 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:33.320916 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:33.321534 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:33.321517 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:33.411757 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:33.411712 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" podStartSLOduration=7.096550179 podStartE2EDuration="21.411700306s" totalImagesPullingTime="14.315150127s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:33.411335348 +0000 UTC m=+21.950058775" watchObservedRunningTime="2026-10-29 23:12:33.411700306 +0000 UTC m=+21.950423734" Oct 29 23:12:33.779743 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:33.779661 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-25m4p" Oct 29 23:12:33.779902 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:33.779788 2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 29 23:12:33.780239 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:33.780218 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-25m4p" Oct 29 23:12:34.059760 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:34.059665 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:34.059932 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:34.059670 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:34.059932 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:34.059802 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dn96d" podUID="b368f5ac-f18a-49d2-b9be-6cca37b7505b" Oct 29 23:12:34.059932 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:34.059902 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xktq" podUID="c6fc8782-2d71-4cb3-ab4b-b1d14aad460f" Oct 29 23:12:34.059932 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:34.059927 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:34.060159 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:34.060003 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gz4mg" podUID="cf41b924-24b7-4a89-9d11-af8b4246fcde" Oct 29 23:12:34.307178 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:34.307151 2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 29 23:12:35.143562 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:35.143530 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gz4mg"] Oct 29 23:12:35.143713 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:35.143643 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:35.143755 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:35.143720 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gz4mg" podUID="cf41b924-24b7-4a89-9d11-af8b4246fcde" Oct 29 23:12:35.146306 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:35.146284 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dn96d"] Oct 29 23:12:35.146397 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:35.146372 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:35.146455 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:35.146441 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dn96d" podUID="b368f5ac-f18a-49d2-b9be-6cca37b7505b" Oct 29 23:12:35.156672 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:35.156651 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2xktq"] Oct 29 23:12:35.156778 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:35.156745 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:35.156844 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:35.156826 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xktq" podUID="c6fc8782-2d71-4cb3-ab4b-b1d14aad460f" Oct 29 23:12:35.308775 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:35.308750 2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 29 23:12:37.059834 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:37.059794 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:37.059834 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:37.059816 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:37.059834 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:37.059840 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:37.060397 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:37.059917 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dn96d" podUID="b368f5ac-f18a-49d2-b9be-6cca37b7505b" Oct 29 23:12:37.060397 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:37.060046 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gz4mg" podUID="cf41b924-24b7-4a89-9d11-af8b4246fcde" Oct 29 23:12:37.060397 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:37.060113 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xktq" podUID="c6fc8782-2d71-4cb3-ab4b-b1d14aad460f" Oct 29 23:12:39.059593 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:39.059561 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:39.060129 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:39.059624 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:39.060129 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:39.059650 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:39.060129 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:39.059740 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xktq" podUID="c6fc8782-2d71-4cb3-ab4b-b1d14aad460f" Oct 29 23:12:39.060129 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:39.059772 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dn96d" podUID="b368f5ac-f18a-49d2-b9be-6cca37b7505b" Oct 29 23:12:39.060129 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:39.059848 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gz4mg" podUID="cf41b924-24b7-4a89-9d11-af8b4246fcde" Oct 29 23:12:41.059827 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.059780 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:41.059827 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.059834 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:41.060310 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.059808 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:41.060310 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.059925 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2xktq" podUID="c6fc8782-2d71-4cb3-ab4b-b1d14aad460f" Oct 29 23:12:41.060310 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.060035 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gz4mg" podUID="cf41b924-24b7-4a89-9d11-af8b4246fcde" Oct 29 23:12:41.060310 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.060091 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dn96d" podUID="b368f5ac-f18a-49d2-b9be-6cca37b7505b" Oct 29 23:12:41.283565 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.283539 2575 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeReady" Oct 29 23:12:41.283742 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.283689 2575 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Oct 29 23:12:41.329988 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.329916 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-z6qn8"] Oct 29 23:12:41.333551 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.333528 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-z6qn8" Oct 29 23:12:41.334006 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.333988 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-74vqh"] Oct 29 23:12:41.336353 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.336331 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-mjz4s"] Oct 29 23:12:41.336457 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.336441 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-74vqh" Oct 29 23:12:41.338422 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.338408 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj"] Oct 29 23:12:41.338556 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.338535 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-mjz4s" Oct 29 23:12:41.340449 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.340432 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-866787b6f9-24krt"] Oct 29 23:12:41.340550 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.340538 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:41.342923 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.342902 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:41.343801 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.343784 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-m72dz"] Oct 29 23:12:41.346238 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.346219 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd"] Oct 29 23:12:41.347671 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.346799 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.352961 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.352942 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm"] Oct 29 23:12:41.353086 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.353066 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" Oct 29 23:12:41.354298 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.354253 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Oct 29 23:12:41.354389 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.354304 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Oct 29 23:12:41.355543 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.355522 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-2gzh7"] Oct 29 23:12:41.355628 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.355597 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:41.358015 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.357997 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pn894"] Oct 29 23:12:41.358150 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.358106 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:41.361548 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.361442 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Oct 29 23:12:41.364191 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.363422 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c"] Oct 29 23:12:41.364289 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.364204 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pn894" Oct 29 23:12:41.366022 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.366000 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5679dbd569-548pj"] Oct 29 23:12:41.366189 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.366169 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" Oct 29 23:12:41.368996 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.368976 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7cbdc4b876-9xjlm"] Oct 29 23:12:41.369103 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.369088 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.372650 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.372632 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-b9jng"] Oct 29 23:12:41.372766 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.372751 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.375714 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.375689 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-nhgxp"] Oct 29 23:12:41.375805 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.375787 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" Oct 29 23:12:41.378697 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.378592 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nhgxp" Oct 29 23:12:41.379305 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.378514 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gnwhl"] Oct 29 23:12:41.382506 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.382483 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-z6qn8"] Oct 29 23:12:41.382607 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.382526 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-t8vj4"] Oct 29 23:12:41.383062 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.382831 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:41.390133 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390113 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-74vqh"] Oct 29 23:12:41.390133 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390126 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-default-certificate\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:41.390299 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390148 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-mjz4s"] Oct 29 23:12:41.390299 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390170 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj"] Oct 29 23:12:41.390299 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390168 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:41.390299 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390201 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd"] Oct 29 23:12:41.390299 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390213 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t8vj4" Oct 29 23:12:41.390299 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390223 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-866787b6f9-24krt"] Oct 29 23:12:41.390299 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390241 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-m72dz"] Oct 29 23:12:41.390299 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390275 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-2gzh7"] Oct 29 23:12:41.390299 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390295 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c"] Oct 29 23:12:41.390705 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390313 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5679dbd569-548pj"] Oct 29 23:12:41.390705 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390307 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swjdw\" (UniqueName: \"kubernetes.io/projected/bf94acd7-605c-4d9a-9021-5ce66a126edd-kube-api-access-swjdw\") pod \"kas-connection-checker-685895c7c9-74vqh\" (UID: \"bf94acd7-605c-4d9a-9021-5ce66a126edd\") " pod="kube-system/kas-connection-checker-685895c7c9-74vqh" Oct 29 23:12:41.390705 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390376 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/63784b03-969b-4e2a-b799-e027ad1c0183-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:41.390705 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390402 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-stats-auth\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:41.390705 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390439 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtg4p\" (UniqueName: \"kubernetes.io/projected/ae184786-7bc6-403a-87fc-185baf65ffa5-kube-api-access-vtg4p\") pod \"kas-connection-checker-685895c7c9-mjz4s\" (UID: \"ae184786-7bc6-403a-87fc-185baf65ffa5\") " pod="kube-system/kas-connection-checker-685895c7c9-mjz4s" Oct 29 23:12:41.390705 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390470 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:41.390705 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390517 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-877j7\" (UniqueName: \"kubernetes.io/projected/5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd-kube-api-access-877j7\") pod \"kas-connection-checker-685895c7c9-z6qn8\" (UID: \"5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd\") " pod="kube-system/kas-connection-checker-685895c7c9-z6qn8" Oct 29 23:12:41.390705 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390584 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:41.390705 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390648 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwkv6\" (UniqueName: \"kubernetes.io/projected/63784b03-969b-4e2a-b799-e027ad1c0183-kube-api-access-zwkv6\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:41.390705 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.390685 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqrk2\" (UniqueName: \"kubernetes.io/projected/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-kube-api-access-xqrk2\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:41.403778 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.403658 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gnwhl"] Oct 29 23:12:41.406942 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.406913 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pn894"] Oct 29 23:12:41.407704 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.407682 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-b9jng"] Oct 29 23:12:41.408379 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.408357 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm"] Oct 29 23:12:41.408934 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.408919 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cbdc4b876-9xjlm"] Oct 29 23:12:41.409691 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.409667 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t8vj4"] Oct 29 23:12:41.411686 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.411663 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-nhgxp"] Oct 29 23:12:41.454703 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.454683 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 29 23:12:41.455053 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.454998 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Oct 29 23:12:41.455117 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.455065 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Oct 29 23:12:41.455117 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.455099 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Oct 29 23:12:41.455200 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.455114 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Oct 29 23:12:41.459079 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459061 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Oct 29 23:12:41.459171 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459114 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Oct 29 23:12:41.459171 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459114 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Oct 29 23:12:41.459171 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459120 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 29 23:12:41.459319 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459299 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 29 23:12:41.459319 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459314 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 29 23:12:41.459419 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459326 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 29 23:12:41.459419 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459341 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 29 23:12:41.459419 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459357 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 29 23:12:41.459419 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459401 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 29 23:12:41.459419 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459409 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Oct 29 23:12:41.459419 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459419 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 29 23:12:41.459656 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459437 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Oct 29 23:12:41.459656 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459451 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Oct 29 23:12:41.459656 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459480 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 29 23:12:41.459656 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459485 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Oct 29 23:12:41.459656 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459509 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 29 23:12:41.459845 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459716 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Oct 29 23:12:41.459845 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459721 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Oct 29 23:12:41.459845 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459734 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 29 23:12:41.459845 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459743 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Oct 29 23:12:41.459845 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.459773 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Oct 29 23:12:41.486074 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486049 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Oct 29 23:12:41.486298 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486111 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Oct 29 23:12:41.486298 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486149 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Oct 29 23:12:41.486462 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486152 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Oct 29 23:12:41.486462 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486421 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Oct 29 23:12:41.486617 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486601 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Oct 29 23:12:41.486662 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486628 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Oct 29 23:12:41.486662 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486635 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Oct 29 23:12:41.486727 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486658 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 29 23:12:41.486825 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486803 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Oct 29 23:12:41.486825 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486814 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Oct 29 23:12:41.486825 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486832 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Oct 29 23:12:41.486825 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486839 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Oct 29 23:12:41.486825 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486845 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Oct 29 23:12:41.486825 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486933 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Oct 29 23:12:41.486825 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486960 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Oct 29 23:12:41.486825 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486974 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 29 23:12:41.486825 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.486964 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Oct 29 23:12:41.487371 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.487020 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 29 23:12:41.487371 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.487036 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-rlkhk\"" Oct 29 23:12:41.487371 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.487095 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-hj644\"" Oct 29 23:12:41.487371 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.487149 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cxqm7\"" Oct 29 23:12:41.487371 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.487335 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4lr6v\"" Oct 29 23:12:41.490971 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.490955 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-877j7\" (UniqueName: \"kubernetes.io/projected/5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd-kube-api-access-877j7\") pod \"kas-connection-checker-685895c7c9-z6qn8\" (UID: \"5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd\") " pod="kube-system/kas-connection-checker-685895c7c9-z6qn8" Oct 29 23:12:41.491038 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.490983 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/837e8971-01d7-44f5-81bb-bf206d191f2d-serving-cert\") pod \"service-ca-operator-544d749c8c-b9jwd\" (UID: \"837e8971-01d7-44f5-81bb-bf206d191f2d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" Oct 29 23:12:41.491038 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491012 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.491038 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491029 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwkv6\" (UniqueName: \"kubernetes.io/projected/63784b03-969b-4e2a-b799-e027ad1c0183-kube-api-access-zwkv6\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:41.491132 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491099 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa19a031-f171-495b-a72d-f4f79b88fceb-cert\") pod \"ingress-canary-t8vj4\" (UID: \"fa19a031-f171-495b-a72d-f4f79b88fceb\") " pod="openshift-ingress-canary/ingress-canary-t8vj4" Oct 29 23:12:41.491211 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491191 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/947dc777-7dd0-4244-913a-5dd068e6e6a3-trusted-ca\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:41.491270 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491244 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrdbd\" (UniqueName: \"kubernetes.io/projected/350f9704-1e0e-4f41-958b-8b0b16761375-kube-api-access-rrdbd\") pod \"volume-data-source-validator-58c9f5f46d-pn894\" (UID: \"350f9704-1e0e-4f41-958b-8b0b16761375\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pn894" Oct 29 23:12:41.491343 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491327 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-certificates\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.491409 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491388 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4tx9\" (UniqueName: \"kubernetes.io/projected/837e8971-01d7-44f5-81bb-bf206d191f2d-kube-api-access-m4tx9\") pod \"service-ca-operator-544d749c8c-b9jwd\" (UID: \"837e8971-01d7-44f5-81bb-bf206d191f2d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" Oct 29 23:12:41.491459 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491445 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:41.491512 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491474 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ad5dcaa4-c226-4639-92fe-ace864d049ac-serving-cert\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.491543 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491519 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-image-registry-private-configuration\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.491602 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.491588 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle podName:b6ad3759-e11c-476d-82ee-fffdf8fd6d47 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:41.991569202 +0000 UTC m=+30.530292614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle") pod "router-default-866787b6f9-24krt" (UID: "b6ad3759-e11c-476d-82ee-fffdf8fd6d47") : configmap references non-existent config key: service-ca.crt Oct 29 23:12:41.491655 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491611 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-bound-sa-token\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.491655 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491633 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-image-registry-private-configuration\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.491780 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491651 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-certificates\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.491780 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491689 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzc9p\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-kube-api-access-bzc9p\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.491780 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491727 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqgl9\" (UniqueName: \"kubernetes.io/projected/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-kube-api-access-vqgl9\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:41.491780 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491765 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad5dcaa4-c226-4639-92fe-ace864d049ac-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.491944 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491788 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-stats-auth\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:41.491944 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491811 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtg4p\" (UniqueName: \"kubernetes.io/projected/ae184786-7bc6-403a-87fc-185baf65ffa5-kube-api-access-vtg4p\") pod \"kas-connection-checker-685895c7c9-mjz4s\" (UID: \"ae184786-7bc6-403a-87fc-185baf65ffa5\") " pod="kube-system/kas-connection-checker-685895c7c9-mjz4s" Oct 29 23:12:41.491944 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491900 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/63784b03-969b-4e2a-b799-e027ad1c0183-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:41.492051 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.491952 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dpmm\" (UniqueName: \"kubernetes.io/projected/947dc777-7dd0-4244-913a-5dd068e6e6a3-kube-api-access-9dpmm\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:41.492108 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.492093 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ad5dcaa4-c226-4639-92fe-ace864d049ac-snapshots\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.492154 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.492116 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b218e5d9-83c0-4976-90da-471e6b86d460-nginx-conf\") pod \"networking-console-plugin-86c478bb88-b9jng\" (UID: \"b218e5d9-83c0-4976-90da-471e6b86d460\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" Oct 29 23:12:41.492154 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.492134 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-b9jng\" (UID: \"b218e5d9-83c0-4976-90da-471e6b86d460\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" Oct 29 23:12:41.492239 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.492226 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:41.492283 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.492248 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2f44\" (UniqueName: \"kubernetes.io/projected/ad5dcaa4-c226-4639-92fe-ace864d049ac-kube-api-access-g2f44\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.492283 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.492279 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/837e8971-01d7-44f5-81bb-bf206d191f2d-config\") pod \"service-ca-operator-544d749c8c-b9jwd\" (UID: \"837e8971-01d7-44f5-81bb-bf206d191f2d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" Oct 29 23:12:41.492363 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.492296 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-ca-trust-extracted\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.492363 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.492355 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 29 23:12:41.492722 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.492687 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:41.492816 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.492741 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:41.492816 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.492781 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/25afb96c-6351-470d-8dfa-dd8a9145be40-ca-trust-extracted\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.492928 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.492818 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.492928 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.492851 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-trusted-ca\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.492928 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.492884 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqrk2\" (UniqueName: \"kubernetes.io/projected/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-kube-api-access-xqrk2\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:41.492928 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.492887 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 29 23:12:41.492928 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.492911 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-config-volume\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:41.492928 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.492919 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/63784b03-969b-4e2a-b799-e027ad1c0183-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:41.493186 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.492947 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs podName:b6ad3759-e11c-476d-82ee-fffdf8fd6d47 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:41.992923619 +0000 UTC m=+30.531647038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs") pod "router-default-866787b6f9-24krt" (UID: "b6ad3759-e11c-476d-82ee-fffdf8fd6d47") : secret "router-metrics-certs-default" not found Oct 29 23:12:41.493186 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.492984 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls podName:63784b03-969b-4e2a-b799-e027ad1c0183 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:41.992965102 +0000 UTC m=+30.531688525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-t6gcj" (UID: "63784b03-969b-4e2a-b799-e027ad1c0183") : secret "cluster-monitoring-operator-tls" not found Oct 29 23:12:41.493186 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.492968 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-tmp-dir\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:41.493186 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493105 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czvnq\" (UniqueName: \"kubernetes.io/projected/8f11a401-ced0-47dc-8814-c23849ba794b-kube-api-access-czvnq\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mc79c\" (UID: \"8f11a401-ced0-47dc-8814-c23849ba794b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" Oct 29 23:12:41.493186 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493146 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-installation-pull-secrets\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.493186 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493182 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-default-certificate\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:41.493491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493217 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnjnb\" (UniqueName: \"kubernetes.io/projected/1575b8c7-d3a3-4956-bb56-72ba26dae4f6-kube-api-access-fnjnb\") pod \"network-check-source-6f78d59f99-nhgxp\" (UID: \"1575b8c7-d3a3-4956-bb56-72ba26dae4f6\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nhgxp" Oct 29 23:12:41.493491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493248 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:41.493491 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493446 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swjdw\" (UniqueName: \"kubernetes.io/projected/bf94acd7-605c-4d9a-9021-5ce66a126edd-kube-api-access-swjdw\") pod \"kas-connection-checker-685895c7c9-74vqh\" (UID: \"bf94acd7-605c-4d9a-9021-5ce66a126edd\") " pod="kube-system/kas-connection-checker-685895c7c9-74vqh" Oct 29 23:12:41.493627 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493518 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-bound-sa-token\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.493627 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493589 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/947dc777-7dd0-4244-913a-5dd068e6e6a3-config\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:41.493627 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493621 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rftvj\" (UniqueName: \"kubernetes.io/projected/fa19a031-f171-495b-a72d-f4f79b88fceb-kube-api-access-rftvj\") pod \"ingress-canary-t8vj4\" (UID: \"fa19a031-f171-495b-a72d-f4f79b88fceb\") " pod="openshift-ingress-canary/ingress-canary-t8vj4" Oct 29 23:12:41.493766 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493652 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-installation-pull-secrets\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.493766 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493687 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:41.493766 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493725 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad5dcaa4-c226-4639-92fe-ace864d049ac-service-ca-bundle\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.493766 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493755 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/947dc777-7dd0-4244-913a-5dd068e6e6a3-serving-cert\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:41.493929 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493800 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad5dcaa4-c226-4639-92fe-ace864d049ac-tmp\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.493929 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493833 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7sk4\" (UniqueName: \"kubernetes.io/projected/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-kube-api-access-m7sk4\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:41.493929 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493870 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f11a401-ced0-47dc-8814-c23849ba794b-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mc79c\" (UID: \"8f11a401-ced0-47dc-8814-c23849ba794b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" Oct 29 23:12:41.493929 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493904 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f11a401-ced0-47dc-8814-c23849ba794b-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mc79c\" (UID: \"8f11a401-ced0-47dc-8814-c23849ba794b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" Oct 29 23:12:41.494089 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493932 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-trusted-ca\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.494089 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.493964 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsr2l\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-kube-api-access-qsr2l\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.498025 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.498001 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-default-certificate\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:41.498940 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.498923 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-stats-auth\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:41.516577 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.516543 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-877j7\" (UniqueName: \"kubernetes.io/projected/5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd-kube-api-access-877j7\") pod \"kas-connection-checker-685895c7c9-z6qn8\" (UID: \"5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd\") " pod="kube-system/kas-connection-checker-685895c7c9-z6qn8" Oct 29 23:12:41.516832 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.516813 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwkv6\" (UniqueName: \"kubernetes.io/projected/63784b03-969b-4e2a-b799-e027ad1c0183-kube-api-access-zwkv6\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:41.516893 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.516842 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtg4p\" (UniqueName: \"kubernetes.io/projected/ae184786-7bc6-403a-87fc-185baf65ffa5-kube-api-access-vtg4p\") pod \"kas-connection-checker-685895c7c9-mjz4s\" (UID: \"ae184786-7bc6-403a-87fc-185baf65ffa5\") " pod="kube-system/kas-connection-checker-685895c7c9-mjz4s" Oct 29 23:12:41.517326 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.517310 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqrk2\" (UniqueName: \"kubernetes.io/projected/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-kube-api-access-xqrk2\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:41.518250 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.518235 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swjdw\" (UniqueName: \"kubernetes.io/projected/bf94acd7-605c-4d9a-9021-5ce66a126edd-kube-api-access-swjdw\") pod \"kas-connection-checker-685895c7c9-74vqh\" (UID: \"bf94acd7-605c-4d9a-9021-5ce66a126edd\") " pod="kube-system/kas-connection-checker-685895c7c9-74vqh" Oct 29 23:12:41.594461 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.594379 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad5dcaa4-c226-4639-92fe-ace864d049ac-service-ca-bundle\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.594461 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.594407 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/947dc777-7dd0-4244-913a-5dd068e6e6a3-serving-cert\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:41.594461 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.594425 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad5dcaa4-c226-4639-92fe-ace864d049ac-tmp\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.594461 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.594439 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7sk4\" (UniqueName: \"kubernetes.io/projected/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-kube-api-access-m7sk4\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:41.594760 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.594726 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f11a401-ced0-47dc-8814-c23849ba794b-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mc79c\" (UID: \"8f11a401-ced0-47dc-8814-c23849ba794b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" Oct 29 23:12:41.594817 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.594771 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f11a401-ced0-47dc-8814-c23849ba794b-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mc79c\" (UID: \"8f11a401-ced0-47dc-8814-c23849ba794b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" Oct 29 23:12:41.594817 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.594776 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad5dcaa4-c226-4639-92fe-ace864d049ac-tmp\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.594817 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.594797 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-trusted-ca\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.594966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.594822 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsr2l\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-kube-api-access-qsr2l\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.594966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.594844 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/837e8971-01d7-44f5-81bb-bf206d191f2d-serving-cert\") pod \"service-ca-operator-544d749c8c-b9jwd\" (UID: \"837e8971-01d7-44f5-81bb-bf206d191f2d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" Oct 29 23:12:41.594966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.594878 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.594966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.594904 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa19a031-f171-495b-a72d-f4f79b88fceb-cert\") pod \"ingress-canary-t8vj4\" (UID: \"fa19a031-f171-495b-a72d-f4f79b88fceb\") " pod="openshift-ingress-canary/ingress-canary-t8vj4" Oct 29 23:12:41.594966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.594927 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/947dc777-7dd0-4244-913a-5dd068e6e6a3-trusted-ca\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:41.594966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.594954 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrdbd\" (UniqueName: \"kubernetes.io/projected/350f9704-1e0e-4f41-958b-8b0b16761375-kube-api-access-rrdbd\") pod \"volume-data-source-validator-58c9f5f46d-pn894\" (UID: \"350f9704-1e0e-4f41-958b-8b0b16761375\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pn894" Oct 29 23:12:41.595239 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.594978 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-certificates\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.595239 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595010 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4tx9\" (UniqueName: \"kubernetes.io/projected/837e8971-01d7-44f5-81bb-bf206d191f2d-kube-api-access-m4tx9\") pod \"service-ca-operator-544d749c8c-b9jwd\" (UID: \"837e8971-01d7-44f5-81bb-bf206d191f2d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" Oct 29 23:12:41.595239 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595061 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ad5dcaa4-c226-4639-92fe-ace864d049ac-serving-cert\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.595239 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595076 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad5dcaa4-c226-4639-92fe-ace864d049ac-service-ca-bundle\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.595239 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595086 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-image-registry-private-configuration\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.595239 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595144 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-bound-sa-token\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.595239 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595175 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-image-registry-private-configuration\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.595239 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595201 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-certificates\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.595239 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595227 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzc9p\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-kube-api-access-bzc9p\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.595696 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595254 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqgl9\" (UniqueName: \"kubernetes.io/projected/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-kube-api-access-vqgl9\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:41.595696 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595311 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad5dcaa4-c226-4639-92fe-ace864d049ac-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.595696 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595348 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dpmm\" (UniqueName: \"kubernetes.io/projected/947dc777-7dd0-4244-913a-5dd068e6e6a3-kube-api-access-9dpmm\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:41.595696 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595379 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ad5dcaa4-c226-4639-92fe-ace864d049ac-snapshots\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.595696 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595391 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f11a401-ced0-47dc-8814-c23849ba794b-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mc79c\" (UID: \"8f11a401-ced0-47dc-8814-c23849ba794b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" Oct 29 23:12:41.595696 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595405 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b218e5d9-83c0-4976-90da-471e6b86d460-nginx-conf\") pod \"networking-console-plugin-86c478bb88-b9jng\" (UID: \"b218e5d9-83c0-4976-90da-471e6b86d460\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" Oct 29 23:12:41.595696 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595450 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-b9jng\" (UID: \"b218e5d9-83c0-4976-90da-471e6b86d460\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" Oct 29 23:12:41.595696 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595504 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2f44\" (UniqueName: \"kubernetes.io/projected/ad5dcaa4-c226-4639-92fe-ace864d049ac-kube-api-access-g2f44\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.595696 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595528 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/837e8971-01d7-44f5-81bb-bf206d191f2d-config\") pod \"service-ca-operator-544d749c8c-b9jwd\" (UID: \"837e8971-01d7-44f5-81bb-bf206d191f2d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" Oct 29 23:12:41.595696 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595550 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-ca-trust-extracted\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.595696 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595583 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:41.595696 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595625 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/25afb96c-6351-470d-8dfa-dd8a9145be40-ca-trust-extracted\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.595696 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595653 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.595696 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595679 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-trusted-ca\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.596360 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595711 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-config-volume\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:41.596360 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595736 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-tmp-dir\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:41.596360 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595762 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czvnq\" (UniqueName: \"kubernetes.io/projected/8f11a401-ced0-47dc-8814-c23849ba794b-kube-api-access-czvnq\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mc79c\" (UID: \"8f11a401-ced0-47dc-8814-c23849ba794b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" Oct 29 23:12:41.596360 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595789 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-installation-pull-secrets\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.596360 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595819 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnjnb\" (UniqueName: \"kubernetes.io/projected/1575b8c7-d3a3-4956-bb56-72ba26dae4f6-kube-api-access-fnjnb\") pod \"network-check-source-6f78d59f99-nhgxp\" (UID: \"1575b8c7-d3a3-4956-bb56-72ba26dae4f6\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nhgxp" Oct 29 23:12:41.596360 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595844 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:41.596360 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595882 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-bound-sa-token\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.596360 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595900 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/947dc777-7dd0-4244-913a-5dd068e6e6a3-trusted-ca\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:41.596360 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595913 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-trusted-ca\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.596360 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595923 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/947dc777-7dd0-4244-913a-5dd068e6e6a3-config\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:41.596360 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595965 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rftvj\" (UniqueName: \"kubernetes.io/projected/fa19a031-f171-495b-a72d-f4f79b88fceb-kube-api-access-rftvj\") pod \"ingress-canary-t8vj4\" (UID: \"fa19a031-f171-495b-a72d-f4f79b88fceb\") " pod="openshift-ingress-canary/ingress-canary-t8vj4" Oct 29 23:12:41.596360 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.595996 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 23:12:41.596360 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.596016 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5679dbd569-548pj: secret "image-registry-tls" not found Oct 29 23:12:41.596360 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.596069 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls podName:6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c nodeName:}" failed. No retries permitted until 2026-10-29 23:12:42.096051068 +0000 UTC m=+30.634774489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls") pod "image-registry-5679dbd569-548pj" (UID: "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c") : secret "image-registry-tls" not found Oct 29 23:12:41.596360 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.596084 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b218e5d9-83c0-4976-90da-471e6b86d460-nginx-conf\") pod \"networking-console-plugin-86c478bb88-b9jng\" (UID: \"b218e5d9-83c0-4976-90da-471e6b86d460\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" Oct 29 23:12:41.597115 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.596512 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/947dc777-7dd0-4244-913a-5dd068e6e6a3-config\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:41.597115 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.596589 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 29 23:12:41.597115 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.596637 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert podName:b218e5d9-83c0-4976-90da-471e6b86d460 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:42.096621222 +0000 UTC m=+30.635344633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-b9jng" (UID: "b218e5d9-83c0-4976-90da-471e6b86d460") : secret "networking-console-plugin-cert" not found Oct 29 23:12:41.597321 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.597282 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/837e8971-01d7-44f5-81bb-bf206d191f2d-config\") pod \"service-ca-operator-544d749c8c-b9jwd\" (UID: \"837e8971-01d7-44f5-81bb-bf206d191f2d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" Oct 29 23:12:41.597407 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.597299 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f11a401-ced0-47dc-8814-c23849ba794b-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mc79c\" (UID: \"8f11a401-ced0-47dc-8814-c23849ba794b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" Oct 29 23:12:41.597407 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.597353 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/947dc777-7dd0-4244-913a-5dd068e6e6a3-serving-cert\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:41.597513 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.597443 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-certificates\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.598802 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.595995 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-installation-pull-secrets\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.598802 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.597645 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:41.598802 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.597674 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/25afb96c-6351-470d-8dfa-dd8a9145be40-ca-trust-extracted\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.598802 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.597795 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 23:12:41.598802 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.597808 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbdc4b876-9xjlm: secret "image-registry-tls" not found Oct 29 23:12:41.598802 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.597814 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-image-registry-private-configuration\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.598802 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.597852 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls podName:25afb96c-6351-470d-8dfa-dd8a9145be40 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:42.097837438 +0000 UTC m=+30.636560858 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls") pod "image-registry-7cbdc4b876-9xjlm" (UID: "25afb96c-6351-470d-8dfa-dd8a9145be40") : secret "image-registry-tls" not found Oct 29 23:12:41.598802 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.597926 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-config-volume\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:41.598802 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.597931 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-ca-trust-extracted\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.598802 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.598212 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 29 23:12:41.598802 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.598474 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-trusted-ca\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.598802 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.598480 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-certificates\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.598802 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.598489 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls podName:8388282a-8b2c-4f5c-ab3d-ec58ea20cc93 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:42.098472473 +0000 UTC m=+30.637195895 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls") pod "dns-default-gnwhl" (UID: "8388282a-8b2c-4f5c-ab3d-ec58ea20cc93") : secret "dns-default-metrics-tls" not found Oct 29 23:12:41.598802 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.598392 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad5dcaa4-c226-4639-92fe-ace864d049ac-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.598802 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.598532 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ad5dcaa4-c226-4639-92fe-ace864d049ac-snapshots\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.598802 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.598551 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-tmp-dir\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:41.598802 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.598746 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 29 23:12:41.599724 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:41.598818 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls podName:8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:42.098803279 +0000 UTC m=+30.637526691 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-jfvvm" (UID: "8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4") : secret "samples-operator-tls" not found Oct 29 23:12:41.599724 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.598822 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-image-registry-private-configuration\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.599724 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.599496 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:41.600382 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.600357 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ad5dcaa4-c226-4639-92fe-ace864d049ac-serving-cert\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.600586 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.600568 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa19a031-f171-495b-a72d-f4f79b88fceb-cert\") pod \"ingress-canary-t8vj4\" (UID: \"fa19a031-f171-495b-a72d-f4f79b88fceb\") " pod="openshift-ingress-canary/ingress-canary-t8vj4" Oct 29 23:12:41.600853 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.600835 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/837e8971-01d7-44f5-81bb-bf206d191f2d-serving-cert\") pod \"service-ca-operator-544d749c8c-b9jwd\" (UID: \"837e8971-01d7-44f5-81bb-bf206d191f2d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" Oct 29 23:12:41.600966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.600951 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-installation-pull-secrets\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.601485 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.601464 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-installation-pull-secrets\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.618467 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.618446 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7sk4\" (UniqueName: \"kubernetes.io/projected/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-kube-api-access-m7sk4\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:41.618844 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.618825 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsr2l\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-kube-api-access-qsr2l\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.623424 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.623396 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-bound-sa-token\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.623642 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.623623 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrdbd\" (UniqueName: \"kubernetes.io/projected/350f9704-1e0e-4f41-958b-8b0b16761375-kube-api-access-rrdbd\") pod \"volume-data-source-validator-58c9f5f46d-pn894\" (UID: \"350f9704-1e0e-4f41-958b-8b0b16761375\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pn894" Oct 29 23:12:41.623769 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.623753 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4tx9\" (UniqueName: \"kubernetes.io/projected/837e8971-01d7-44f5-81bb-bf206d191f2d-kube-api-access-m4tx9\") pod \"service-ca-operator-544d749c8c-b9jwd\" (UID: \"837e8971-01d7-44f5-81bb-bf206d191f2d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" Oct 29 23:12:41.624958 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.624942 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rftvj\" (UniqueName: \"kubernetes.io/projected/fa19a031-f171-495b-a72d-f4f79b88fceb-kube-api-access-rftvj\") pod \"ingress-canary-t8vj4\" (UID: \"fa19a031-f171-495b-a72d-f4f79b88fceb\") " pod="openshift-ingress-canary/ingress-canary-t8vj4" Oct 29 23:12:41.628920 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.628857 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqgl9\" (UniqueName: \"kubernetes.io/projected/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-kube-api-access-vqgl9\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:41.629368 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.629341 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czvnq\" (UniqueName: \"kubernetes.io/projected/8f11a401-ced0-47dc-8814-c23849ba794b-kube-api-access-czvnq\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mc79c\" (UID: \"8f11a401-ced0-47dc-8814-c23849ba794b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" Oct 29 23:12:41.629485 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.629466 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzc9p\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-kube-api-access-bzc9p\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:41.629830 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.629809 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnjnb\" (UniqueName: \"kubernetes.io/projected/1575b8c7-d3a3-4956-bb56-72ba26dae4f6-kube-api-access-fnjnb\") pod \"network-check-source-6f78d59f99-nhgxp\" (UID: \"1575b8c7-d3a3-4956-bb56-72ba26dae4f6\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nhgxp" Oct 29 23:12:41.629969 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.629949 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dpmm\" (UniqueName: \"kubernetes.io/projected/947dc777-7dd0-4244-913a-5dd068e6e6a3-kube-api-access-9dpmm\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:41.631137 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.631115 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2f44\" (UniqueName: \"kubernetes.io/projected/ad5dcaa4-c226-4639-92fe-ace864d049ac-kube-api-access-g2f44\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.632323 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.632295 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-bound-sa-token\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:41.647938 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.647916 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-z6qn8" Oct 29 23:12:41.657090 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.657065 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-74vqh" Oct 29 23:12:41.667441 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.667419 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-mjz4s" Oct 29 23:12:41.684503 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.684464 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:41.690701 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.690672 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:41.703994 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.703903 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" Oct 29 23:12:41.719143 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.718948 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:41.726511 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.726354 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pn894" Oct 29 23:12:41.759203 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.745396 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" Oct 29 23:12:41.759203 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.758109 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nhgxp" Oct 29 23:12:41.769095 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.767349 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t8vj4" Oct 29 23:12:41.905495 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.905460 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:41.988384 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:41.988358 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-mjz4s"] Oct 29 23:12:41.990867 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:41.990843 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae184786_7bc6_403a_87fc_185baf65ffa5.slice/crio-e4cbf370e140b1541199154f795361d6b0d7b53f23bc73b3585577e1c9f6fd0e WatchSource:0}: Error finding container e4cbf370e140b1541199154f795361d6b0d7b53f23bc73b3585577e1c9f6fd0e: Status 404 returned error can't find the container with id e4cbf370e140b1541199154f795361d6b0d7b53f23bc73b3585577e1c9f6fd0e Oct 29 23:12:42.000056 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.000013 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-74vqh"] Oct 29 23:12:42.001981 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:42.001949 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf94acd7_605c_4d9a_9021_5ce66a126edd.slice/crio-a2a8836701e6ddf02d832bc5bd6c5d5964e776520c2c9cec1a5f49d7e98774cd WatchSource:0}: Error finding container a2a8836701e6ddf02d832bc5bd6c5d5964e776520c2c9cec1a5f49d7e98774cd: Status 404 returned error can't find the container with id a2a8836701e6ddf02d832bc5bd6c5d5964e776520c2c9cec1a5f49d7e98774cd Oct 29 23:12:42.003279 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.003244 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:42.003360 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.003330 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:42.003416 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:42.003379 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle podName:b6ad3759-e11c-476d-82ee-fffdf8fd6d47 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:43.003360598 +0000 UTC m=+31.542084021 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle") pod "router-default-866787b6f9-24krt" (UID: "b6ad3759-e11c-476d-82ee-fffdf8fd6d47") : configmap references non-existent config key: service-ca.crt Oct 29 23:12:42.003416 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.003401 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:42.003502 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:42.003431 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 29 23:12:42.003502 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:42.003472 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls podName:63784b03-969b-4e2a-b799-e027ad1c0183 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:43.003459261 +0000 UTC m=+31.542182670 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-t6gcj" (UID: "63784b03-969b-4e2a-b799-e027ad1c0183") : secret "cluster-monitoring-operator-tls" not found Oct 29 23:12:42.003563 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:42.003515 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 29 23:12:42.003563 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:42.003549 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs podName:b6ad3759-e11c-476d-82ee-fffdf8fd6d47 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:43.003537664 +0000 UTC m=+31.542261085 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs") pod "router-default-866787b6f9-24krt" (UID: "b6ad3759-e11c-476d-82ee-fffdf8fd6d47") : secret "router-metrics-certs-default" not found Oct 29 23:12:42.010924 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.010901 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-z6qn8"] Oct 29 23:12:42.012246 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:42.012219 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d039e77_d1dc_4215_9dd0_2a95e6d7e8fd.slice/crio-15fd926f7c91f01a1e15e40ea2b2d3c9949fad4f8bed6f1cec57a3c827730f16 WatchSource:0}: Error finding container 15fd926f7c91f01a1e15e40ea2b2d3c9949fad4f8bed6f1cec57a3c827730f16: Status 404 returned error can't find the container with id 15fd926f7c91f01a1e15e40ea2b2d3c9949fad4f8bed6f1cec57a3c827730f16 Oct 29 23:12:42.024688 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.024610 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-m72dz"] Oct 29 23:12:42.031442 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:42.031368 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad5dcaa4_c226_4639_92fe_ace864d049ac.slice/crio-ff4dfe98edc9d895bbc191f3efdfe4fd60c8092d4a11d6abf41b31fcaff7ebf5 WatchSource:0}: Error finding container ff4dfe98edc9d895bbc191f3efdfe4fd60c8092d4a11d6abf41b31fcaff7ebf5: Status 404 returned error can't find the container with id ff4dfe98edc9d895bbc191f3efdfe4fd60c8092d4a11d6abf41b31fcaff7ebf5 Oct 29 23:12:42.076233 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.076199 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd"] Oct 29 23:12:42.077297 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.077273 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-2gzh7"] Oct 29 23:12:42.091425 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:42.091403 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod947dc777_7dd0_4244_913a_5dd068e6e6a3.slice/crio-82353c5785c533809f1de6dd9818f418c39cccf47e5cde5e41675d3cd4767a9f WatchSource:0}: Error finding container 82353c5785c533809f1de6dd9818f418c39cccf47e5cde5e41675d3cd4767a9f: Status 404 returned error can't find the container with id 82353c5785c533809f1de6dd9818f418c39cccf47e5cde5e41675d3cd4767a9f Oct 29 23:12:42.104620 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.101337 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pn894"] Oct 29 23:12:42.104620 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.104583 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-b9jng\" (UID: \"b218e5d9-83c0-4976-90da-471e6b86d460\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" Oct 29 23:12:42.104909 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.104640 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:42.104909 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.104678 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:42.104909 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.104713 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:42.104909 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.104793 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:42.105092 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:42.104928 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 23:12:42.105092 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:42.104942 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5679dbd569-548pj: secret "image-registry-tls" not found Oct 29 23:12:42.105092 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:42.105003 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls podName:6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c nodeName:}" failed. No retries permitted until 2026-10-29 23:12:43.104983982 +0000 UTC m=+31.643707405 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls") pod "image-registry-5679dbd569-548pj" (UID: "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c") : secret "image-registry-tls" not found Oct 29 23:12:42.105092 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:42.105088 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 29 23:12:42.105389 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:42.105126 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert podName:b218e5d9-83c0-4976-90da-471e6b86d460 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:43.105113209 +0000 UTC m=+31.643836629 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-b9jng" (UID: "b218e5d9-83c0-4976-90da-471e6b86d460") : secret "networking-console-plugin-cert" not found Oct 29 23:12:42.105389 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:42.105184 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 29 23:12:42.105389 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:42.105217 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls podName:8388282a-8b2c-4f5c-ab3d-ec58ea20cc93 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:43.105206125 +0000 UTC m=+31.643929532 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls") pod "dns-default-gnwhl" (UID: "8388282a-8b2c-4f5c-ab3d-ec58ea20cc93") : secret "dns-default-metrics-tls" not found Oct 29 23:12:42.105389 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:42.105353 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 23:12:42.105389 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:42.105368 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbdc4b876-9xjlm: secret "image-registry-tls" not found Oct 29 23:12:42.105641 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:42.105414 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls podName:25afb96c-6351-470d-8dfa-dd8a9145be40 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:43.105400961 +0000 UTC m=+31.644124380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls") pod "image-registry-7cbdc4b876-9xjlm" (UID: "25afb96c-6351-470d-8dfa-dd8a9145be40") : secret "image-registry-tls" not found Oct 29 23:12:42.105641 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:42.105428 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 29 23:12:42.105641 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:42.105486 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls podName:8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:43.105468526 +0000 UTC m=+31.644191950 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-jfvvm" (UID: "8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4") : secret "samples-operator-tls" not found Oct 29 23:12:42.113246 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:42.113217 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod350f9704_1e0e_4f41_958b_8b0b16761375.slice/crio-141e20972e4b1ee800df8a6bdcb140a79bc979449f8cb50efcf252d7f2566176 WatchSource:0}: Error finding container 141e20972e4b1ee800df8a6bdcb140a79bc979449f8cb50efcf252d7f2566176: Status 404 returned error can't find the container with id 141e20972e4b1ee800df8a6bdcb140a79bc979449f8cb50efcf252d7f2566176 Oct 29 23:12:42.137167 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.137144 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c"] Oct 29 23:12:42.141548 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.141513 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t8vj4"] Oct 29 23:12:42.144496 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.144470 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:42.145465 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.145442 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-nhgxp"] Oct 29 23:12:42.146394 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:42.146366 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa19a031_f171_495b_a72d_f4f79b88fceb.slice/crio-e2f99c191ef1a42bf8aee30eaaa041e3a03f7536a34173bd03dc701128e6b7a7 WatchSource:0}: Error finding container e2f99c191ef1a42bf8aee30eaaa041e3a03f7536a34173bd03dc701128e6b7a7: Status 404 returned error can't find the container with id e2f99c191ef1a42bf8aee30eaaa041e3a03f7536a34173bd03dc701128e6b7a7 Oct 29 23:12:42.147855 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:42.147813 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1575b8c7_d3a3_4956_bb56_72ba26dae4f6.slice/crio-baedcc53bf54f0b96ed8967ff1465e6d9552452fdfc48192e1b26666f1976295 WatchSource:0}: Error finding container baedcc53bf54f0b96ed8967ff1465e6d9552452fdfc48192e1b26666f1976295: Status 404 returned error can't find the container with id baedcc53bf54f0b96ed8967ff1465e6d9552452fdfc48192e1b26666f1976295 Oct 29 23:12:42.150632 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:42.150443 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f11a401_ced0_47dc_8814_c23849ba794b.slice/crio-ce41cdd0cdfb13694d7dcafdb1f3ce7ec1f1caf602a4d21511fdef6c0af294fd WatchSource:0}: Error finding container ce41cdd0cdfb13694d7dcafdb1f3ce7ec1f1caf602a4d21511fdef6c0af294fd: Status 404 returned error can't find the container with id ce41cdd0cdfb13694d7dcafdb1f3ce7ec1f1caf602a4d21511fdef6c0af294fd Oct 29 23:12:42.303639 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.303606 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:42.324562 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.324530 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" event={"ID":"947dc777-7dd0-4244-913a-5dd068e6e6a3","Type":"ContainerStarted","Data":"82353c5785c533809f1de6dd9818f418c39cccf47e5cde5e41675d3cd4767a9f"} Oct 29 23:12:42.325761 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.325740 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t8vj4" event={"ID":"fa19a031-f171-495b-a72d-f4f79b88fceb","Type":"ContainerStarted","Data":"e2f99c191ef1a42bf8aee30eaaa041e3a03f7536a34173bd03dc701128e6b7a7"} Oct 29 23:12:42.326889 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.326866 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" event={"ID":"8f11a401-ced0-47dc-8814-c23849ba794b","Type":"ContainerStarted","Data":"ce41cdd0cdfb13694d7dcafdb1f3ce7ec1f1caf602a4d21511fdef6c0af294fd"} Oct 29 23:12:42.328005 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.327984 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-m72dz" event={"ID":"ad5dcaa4-c226-4639-92fe-ace864d049ac","Type":"ContainerStarted","Data":"ff4dfe98edc9d895bbc191f3efdfe4fd60c8092d4a11d6abf41b31fcaff7ebf5"} Oct 29 23:12:42.328977 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.328959 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pn894" event={"ID":"350f9704-1e0e-4f41-958b-8b0b16761375","Type":"ContainerStarted","Data":"141e20972e4b1ee800df8a6bdcb140a79bc979449f8cb50efcf252d7f2566176"} Oct 29 23:12:42.329962 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.329945 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" event={"ID":"837e8971-01d7-44f5-81bb-bf206d191f2d","Type":"ContainerStarted","Data":"fbe63081c629c2739579aaea9f2d9cd911ded08084074e23ea18ea78aac22f30"} Oct 29 23:12:42.331325 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.331306 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-mjz4s" event={"ID":"ae184786-7bc6-403a-87fc-185baf65ffa5","Type":"ContainerStarted","Data":"096f3a4e7fd646f9732069ee9b0f8fda15bd5e543cff6551491ec075f9d25115"} Oct 29 23:12:42.331399 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.331336 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-mjz4s" event={"ID":"ae184786-7bc6-403a-87fc-185baf65ffa5","Type":"ContainerStarted","Data":"e4cbf370e140b1541199154f795361d6b0d7b53f23bc73b3585577e1c9f6fd0e"} Oct 29 23:12:42.332628 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.332602 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-74vqh" event={"ID":"bf94acd7-605c-4d9a-9021-5ce66a126edd","Type":"ContainerStarted","Data":"b0f1e0e38a2624a9c5deff16dd8d9754e4742691501113bd3742dc3a4bdf8f60"} Oct 29 23:12:42.332692 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.332633 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-74vqh" event={"ID":"bf94acd7-605c-4d9a-9021-5ce66a126edd","Type":"ContainerStarted","Data":"a2a8836701e6ddf02d832bc5bd6c5d5964e776520c2c9cec1a5f49d7e98774cd"} Oct 29 23:12:42.333878 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.333858 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nhgxp" event={"ID":"1575b8c7-d3a3-4956-bb56-72ba26dae4f6","Type":"ContainerStarted","Data":"baedcc53bf54f0b96ed8967ff1465e6d9552452fdfc48192e1b26666f1976295"} Oct 29 23:12:42.335103 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.335081 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-z6qn8" event={"ID":"5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd","Type":"ContainerStarted","Data":"274f0d6c0851af2770871fd33d620f3e91fa56e3eaad0caa770e8e4b96df8b13"} Oct 29 23:12:42.335181 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.335112 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-z6qn8" event={"ID":"5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd","Type":"ContainerStarted","Data":"15fd926f7c91f01a1e15e40ea2b2d3c9949fad4f8bed6f1cec57a3c827730f16"} Oct 29 23:12:42.373202 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.373125 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-mjz4s" podStartSLOduration=2083.37311351 podStartE2EDuration="34m43.37311351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:37:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:42.372547985 +0000 UTC m=+30.911271414" watchObservedRunningTime="2026-10-29 23:12:42.37311351 +0000 UTC m=+30.911836937" Oct 29 23:12:42.417037 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.416973 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-74vqh" podStartSLOduration=2083.416959371 podStartE2EDuration="34m43.416959371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:37:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:42.416182336 +0000 UTC m=+30.954905763" watchObservedRunningTime="2026-10-29 23:12:42.416959371 +0000 UTC m=+30.955682799" Oct 29 23:12:42.457127 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.457073 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-z6qn8" podStartSLOduration=2083.457056617 podStartE2EDuration="34m43.457056617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:37:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:42.456896649 +0000 UTC m=+30.995620078" watchObservedRunningTime="2026-10-29 23:12:42.457056617 +0000 UTC m=+30.995780046" Oct 29 23:12:42.458603 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:42.458576 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:43.013352 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:43.013311 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:43.013553 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:43.013408 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:43.013553 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:43.013454 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:43.013696 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:43.013679 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 29 23:12:43.013760 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:43.013749 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs podName:b6ad3759-e11c-476d-82ee-fffdf8fd6d47 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:45.013729754 +0000 UTC m=+33.552453177 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs") pod "router-default-866787b6f9-24krt" (UID: "b6ad3759-e11c-476d-82ee-fffdf8fd6d47") : secret "router-metrics-certs-default" not found Oct 29 23:12:43.013836 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:43.013824 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle podName:b6ad3759-e11c-476d-82ee-fffdf8fd6d47 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:45.013813499 +0000 UTC m=+33.552536919 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle") pod "router-default-866787b6f9-24krt" (UID: "b6ad3759-e11c-476d-82ee-fffdf8fd6d47") : configmap references non-existent config key: service-ca.crt Oct 29 23:12:43.013898 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:43.013887 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 29 23:12:43.013945 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:43.013919 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls podName:63784b03-969b-4e2a-b799-e027ad1c0183 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:45.013908314 +0000 UTC m=+33.552631723 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-t6gcj" (UID: "63784b03-969b-4e2a-b799-e027ad1c0183") : secret "cluster-monitoring-operator-tls" not found Oct 29 23:12:43.059582 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:43.059422 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:43.061770 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:43.059929 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:43.061770 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:43.060777 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:43.078063 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:43.077556 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 29 23:12:43.078063 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:43.077741 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-m6t5c\"" Oct 29 23:12:43.078063 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:43.077920 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 29 23:12:43.078063 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:43.077990 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wscf4\"" Oct 29 23:12:43.078063 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:43.078031 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-z28mk\"" Oct 29 23:12:43.119888 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:43.119833 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-b9jng\" (UID: \"b218e5d9-83c0-4976-90da-471e6b86d460\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" Oct 29 23:12:43.120077 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:43.119911 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:43.120077 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:43.119948 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:43.120077 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:43.119983 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:43.120077 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:43.119997 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 29 23:12:43.125920 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:43.125314 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:43.125920 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:43.125440 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert podName:b218e5d9-83c0-4976-90da-471e6b86d460 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:45.125416345 +0000 UTC m=+33.664139765 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-b9jng" (UID: "b218e5d9-83c0-4976-90da-471e6b86d460") : secret "networking-console-plugin-cert" not found Oct 29 23:12:43.125920 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:43.125551 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 23:12:43.125920 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:43.125564 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbdc4b876-9xjlm: secret "image-registry-tls" not found Oct 29 23:12:43.125920 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:43.125623 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls podName:25afb96c-6351-470d-8dfa-dd8a9145be40 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:45.125611302 +0000 UTC m=+33.664334722 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls") pod "image-registry-7cbdc4b876-9xjlm" (UID: "25afb96c-6351-470d-8dfa-dd8a9145be40") : secret "image-registry-tls" not found Oct 29 23:12:43.125920 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:43.125696 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 29 23:12:43.125920 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:43.125734 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls podName:8388282a-8b2c-4f5c-ab3d-ec58ea20cc93 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:45.125723014 +0000 UTC m=+33.664446437 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls") pod "dns-default-gnwhl" (UID: "8388282a-8b2c-4f5c-ab3d-ec58ea20cc93") : secret "dns-default-metrics-tls" not found Oct 29 23:12:43.125920 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:43.125793 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 29 23:12:43.125920 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:43.125830 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls podName:8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:45.125820203 +0000 UTC m=+33.664543626 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-jfvvm" (UID: "8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4") : secret "samples-operator-tls" not found Oct 29 23:12:43.127324 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:43.127131 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 23:12:43.127324 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:43.127159 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5679dbd569-548pj: secret "image-registry-tls" not found Oct 29 23:12:43.127324 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:43.127223 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls podName:6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c nodeName:}" failed. No retries permitted until 2026-10-29 23:12:45.127201151 +0000 UTC m=+33.665924575 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls") pod "image-registry-5679dbd569-548pj" (UID: "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c") : secret "image-registry-tls" not found Oct 29 23:12:44.009417 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:44.007709 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:44.009417 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:44.007980 2575 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 29 23:12:44.035762 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:44.034689 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:44.250030 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:44.249900 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:44.344845 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:44.344768 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pn894" event={"ID":"350f9704-1e0e-4f41-958b-8b0b16761375","Type":"ContainerStarted","Data":"70bb2290dcdae778112eb87e0c996f619dc1ff41724a88b78398dff940fd1007"} Oct 29 23:12:44.407189 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:44.407154 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:44.578349 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:44.578196 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:44.744788 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:44.744716 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:44.898465 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:44.898050 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:45.046097 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:45.045105 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:45.046097 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:45.045195 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:45.046097 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:45.045243 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:45.046097 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.045479 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 29 23:12:45.046097 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.045543 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs podName:b6ad3759-e11c-476d-82ee-fffdf8fd6d47 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:49.045524529 +0000 UTC m=+37.584247949 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs") pod "router-default-866787b6f9-24krt" (UID: "b6ad3759-e11c-476d-82ee-fffdf8fd6d47") : secret "router-metrics-certs-default" not found Oct 29 23:12:45.046097 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.045954 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle podName:b6ad3759-e11c-476d-82ee-fffdf8fd6d47 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:49.045938408 +0000 UTC m=+37.584661828 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle") pod "router-default-866787b6f9-24krt" (UID: "b6ad3759-e11c-476d-82ee-fffdf8fd6d47") : configmap references non-existent config key: service-ca.crt Oct 29 23:12:45.046097 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.046024 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 29 23:12:45.046097 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.046061 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls podName:63784b03-969b-4e2a-b799-e027ad1c0183 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:49.046048587 +0000 UTC m=+37.584772062 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-t6gcj" (UID: "63784b03-969b-4e2a-b799-e027ad1c0183") : secret "cluster-monitoring-operator-tls" not found Oct 29 23:12:45.146745 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:45.146711 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-b9jng\" (UID: \"b218e5d9-83c0-4976-90da-471e6b86d460\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" Oct 29 23:12:45.146896 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:45.146772 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:45.146896 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:45.146808 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:45.146896 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:45.146835 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:45.146896 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:45.146888 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:45.147112 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.147028 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 23:12:45.147112 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.147040 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5679dbd569-548pj: secret "image-registry-tls" not found Oct 29 23:12:45.147112 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.147084 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls podName:6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c nodeName:}" failed. No retries permitted until 2026-10-29 23:12:49.147070388 +0000 UTC m=+37.685793808 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls") pod "image-registry-5679dbd569-548pj" (UID: "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c") : secret "image-registry-tls" not found Oct 29 23:12:45.147483 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.147421 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 23:12:45.147483 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.147445 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbdc4b876-9xjlm: secret "image-registry-tls" not found Oct 29 23:12:45.147483 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.147457 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 29 23:12:45.147668 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.147458 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 29 23:12:45.147668 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.147502 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls podName:25afb96c-6351-470d-8dfa-dd8a9145be40 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:49.147481787 +0000 UTC m=+37.686205212 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls") pod "image-registry-7cbdc4b876-9xjlm" (UID: "25afb96c-6351-470d-8dfa-dd8a9145be40") : secret "image-registry-tls" not found Oct 29 23:12:45.147668 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.147518 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 29 23:12:45.147668 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.147521 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls podName:8388282a-8b2c-4f5c-ab3d-ec58ea20cc93 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:49.147511605 +0000 UTC m=+37.686235016 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls") pod "dns-default-gnwhl" (UID: "8388282a-8b2c-4f5c-ab3d-ec58ea20cc93") : secret "dns-default-metrics-tls" not found Oct 29 23:12:45.147668 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.147577 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert podName:b218e5d9-83c0-4976-90da-471e6b86d460 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:49.147564921 +0000 UTC m=+37.686288334 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-b9jng" (UID: "b218e5d9-83c0-4976-90da-471e6b86d460") : secret "networking-console-plugin-cert" not found Oct 29 23:12:45.147668 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.147589 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls podName:8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:49.147582941 +0000 UTC m=+37.686306348 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-jfvvm" (UID: "8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4") : secret "samples-operator-tls" not found Oct 29 23:12:45.214228 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:45.214185 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:45.393605 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:45.393532 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:45.752425 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:45.752391 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:45.752545 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:45.752464 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:45.752596 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.752568 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 29 23:12:45.752655 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:45.752647 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs podName:c6fc8782-2d71-4cb3-ab4b-b1d14aad460f nodeName:}" failed. No retries permitted until 2026-10-29 23:13:17.752626273 +0000 UTC m=+66.291349685 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs") pod "network-metrics-daemon-2xktq" (UID: "c6fc8782-2d71-4cb3-ab4b-b1d14aad460f") : secret "metrics-daemon-secret" not found Oct 29 23:12:45.760220 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:45.760176 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:45.795372 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:45.795338 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:45.853246 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:45.853211 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4mr2\" (UniqueName: \"kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2\") pod \"network-check-target-dn96d\" (UID: \"b368f5ac-f18a-49d2-b9be-6cca37b7505b\") " pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:45.857058 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:45.857030 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4mr2\" (UniqueName: \"kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2\") pod \"network-check-target-dn96d\" (UID: \"b368f5ac-f18a-49d2-b9be-6cca37b7505b\") " pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:46.077581 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.077388 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:46.122426 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.121863 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pn894" podStartSLOduration=2058.291077745 podStartE2EDuration="34m20.121846417s" totalImagesPullingTime="1.830768672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:44.388064144 +0000 UTC m=+32.926787573" watchObservedRunningTime="2026-10-29 23:12:46.121846417 +0000 UTC m=+34.660569846" Oct 29 23:12:46.122942 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.122920 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gz4mg"] Oct 29 23:12:46.354571 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.354489 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" event={"ID":"8f11a401-ced0-47dc-8814-c23849ba794b","Type":"ContainerStarted","Data":"14bd2f8d9b7b87ac8e1a34b97d52be9fa02320b64500d0a6834d21ae65b7d9fb"} Oct 29 23:12:46.358290 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.357130 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-m72dz" event={"ID":"ad5dcaa4-c226-4639-92fe-ace864d049ac","Type":"ContainerStarted","Data":"ba6a74650dec9da409edd2d622173de72d3a67137ca3f0d1417d535ce7437ab7"} Oct 29 23:12:46.360909 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.360883 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" event={"ID":"837e8971-01d7-44f5-81bb-bf206d191f2d","Type":"ContainerStarted","Data":"de99a0ae73dca6e7394428edade9d95365eb9c5224a7af6940d5e72610e7e6d8"} Oct 29 23:12:46.363373 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.363355 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/0.log" Oct 29 23:12:46.363459 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.363389 2575 generic.go:356] "Generic (PLEG): container finished" podID="947dc777-7dd0-4244-913a-5dd068e6e6a3" containerID="37e3534ab94e2e9ef40efb03a621dbb8b1cef1ca927588c7a9ebdd7f511fdbf6" exitCode=255 Oct 29 23:12:46.363459 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.363452 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" event={"ID":"947dc777-7dd0-4244-913a-5dd068e6e6a3","Type":"ContainerDied","Data":"37e3534ab94e2e9ef40efb03a621dbb8b1cef1ca927588c7a9ebdd7f511fdbf6"} Oct 29 23:12:46.363718 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.363702 2575 scope.go:122] "RemoveContainer" containerID="37e3534ab94e2e9ef40efb03a621dbb8b1cef1ca927588c7a9ebdd7f511fdbf6" Oct 29 23:12:46.367637 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.367604 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gz4mg" event={"ID":"cf41b924-24b7-4a89-9d11-af8b4246fcde","Type":"ContainerStarted","Data":"13ed2c84b37ccdceedcaa24fb57881ee76e650cdf5ba3719e10e76572a6873d8"} Oct 29 23:12:46.403310 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.400914 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" podStartSLOduration=2055.506828023 podStartE2EDuration="34m19.400896367s" totalImagesPullingTime="3.894068344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:38:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:46.40067192 +0000 UTC m=+34.939395348" watchObservedRunningTime="2026-10-29 23:12:46.400896367 +0000 UTC m=+34.939619795" Oct 29 23:12:46.404950 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.404922 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dn96d"] Oct 29 23:12:46.410633 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:46.409785 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb368f5ac_f18a_49d2_b9be_6cca37b7505b.slice/crio-e008b9fb8b4d2bb09c28afa47ec76dad17bd4525da1dc9650d2142c524970493 WatchSource:0}: Error finding container e008b9fb8b4d2bb09c28afa47ec76dad17bd4525da1dc9650d2142c524970493: Status 404 returned error can't find the container with id e008b9fb8b4d2bb09c28afa47ec76dad17bd4525da1dc9650d2142c524970493 Oct 29 23:12:46.447624 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.447575 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-m72dz" podStartSLOduration=2056.753183369 podStartE2EDuration="34m20.447557804s" totalImagesPullingTime="3.694374435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:46.447418417 +0000 UTC m=+34.986141845" watchObservedRunningTime="2026-10-29 23:12:46.447557804 +0000 UTC m=+34.986281234" Oct 29 23:12:46.552044 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.551960 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" podStartSLOduration=2056.46330863 podStartE2EDuration="34m20.551931079s" totalImagesPullingTime="4.088622449s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:38:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:46.498971707 +0000 UTC m=+35.037695136" watchObservedRunningTime="2026-10-29 23:12:46.551931079 +0000 UTC m=+35.090654507" Oct 29 23:12:46.768708 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.767252 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-x975f"] Oct 29 23:12:46.799112 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.798371 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x975f"] Oct 29 23:12:46.799112 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.798473 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:46.815291 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.815082 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 29 23:12:46.815291 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.815221 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 29 23:12:46.815481 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.815397 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-88spp\"" Oct 29 23:12:46.833044 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.833011 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-hhksv-vlwlb"] Oct 29 23:12:46.855127 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.855087 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:46.862216 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.862189 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-hhksv-vlwlb"] Oct 29 23:12:46.905489 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.905453 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:46.919531 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.915508 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lbwxf\"" Oct 29 23:12:46.978805 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.978765 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:46.978966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.978812 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shjmc\" (UniqueName: \"kubernetes.io/projected/fce92e88-114c-49c7-b658-193d508126e9-kube-api-access-shjmc\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:46.978966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.978843 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rdlx\" (UniqueName: \"kubernetes.io/projected/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-api-access-9rdlx\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:46.978966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.978884 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3b53bf9-0305-43ad-99b1-26f35c841020-data-volume\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:46.978966 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.978960 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:46.979198 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.978988 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-service-ca-bundle\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:46.979198 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.979063 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b3b53bf9-0305-43ad-99b1-26f35c841020-crio-socket\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:46.979198 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.979086 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fce92e88-114c-49c7-b658-193d508126e9-archives-path\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:46.979198 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:46.979109 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-trusted-ca-bundle\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:47.057516 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.057441 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:47.081401 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.080472 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:47.081401 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.080529 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-service-ca-bundle\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:47.081401 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.080613 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b3b53bf9-0305-43ad-99b1-26f35c841020-crio-socket\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:47.081401 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.080636 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fce92e88-114c-49c7-b658-193d508126e9-archives-path\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:47.081401 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.080661 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-trusted-ca-bundle\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:47.081401 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.080731 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:47.081401 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.080767 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shjmc\" (UniqueName: \"kubernetes.io/projected/fce92e88-114c-49c7-b658-193d508126e9-kube-api-access-shjmc\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:47.081401 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.080794 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rdlx\" (UniqueName: \"kubernetes.io/projected/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-api-access-9rdlx\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:47.081401 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.080837 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3b53bf9-0305-43ad-99b1-26f35c841020-data-volume\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:47.081401 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.081296 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3b53bf9-0305-43ad-99b1-26f35c841020-data-volume\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:47.081401 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:47.081355 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 29 23:12:47.082175 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:47.081433 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm podName:b3b53bf9-0305-43ad-99b1-26f35c841020 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:47.581411681 +0000 UTC m=+36.120135108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x975f" (UID: "b3b53bf9-0305-43ad-99b1-26f35c841020") : configmap "kube-rbac-proxy" not found Oct 29 23:12:47.082175 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:47.081706 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 29 23:12:47.082175 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:47.081756 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls podName:b3b53bf9-0305-43ad-99b1-26f35c841020 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:47.581740083 +0000 UTC m=+36.120463504 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x975f" (UID: "b3b53bf9-0305-43ad-99b1-26f35c841020") : secret "insights-runtime-extractor-tls" not found Oct 29 23:12:47.082175 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.082025 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-trusted-ca-bundle\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:47.082175 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.082081 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b3b53bf9-0305-43ad-99b1-26f35c841020-crio-socket\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:47.082444 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.082232 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-service-ca-bundle\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:47.082444 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.082281 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fce92e88-114c-49c7-b658-193d508126e9-archives-path\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:47.109579 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.109543 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shjmc\" (UniqueName: \"kubernetes.io/projected/fce92e88-114c-49c7-b658-193d508126e9-kube-api-access-shjmc\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:47.113221 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.113194 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rdlx\" (UniqueName: \"kubernetes.io/projected/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-api-access-9rdlx\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:47.196369 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.196334 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:47.266219 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.266128 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:47.376518 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.376444 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/1.log" Oct 29 23:12:47.377243 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.377212 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/0.log" Oct 29 23:12:47.377381 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.377321 2575 generic.go:356] "Generic (PLEG): container finished" podID="947dc777-7dd0-4244-913a-5dd068e6e6a3" containerID="a32678eaae6ae6d7b889b7350cba11dc4d854c7dae6d8bfaa12ca78621468dbb" exitCode=255 Oct 29 23:12:47.377684 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.377659 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" event={"ID":"947dc777-7dd0-4244-913a-5dd068e6e6a3","Type":"ContainerDied","Data":"a32678eaae6ae6d7b889b7350cba11dc4d854c7dae6d8bfaa12ca78621468dbb"} Oct 29 23:12:47.377744 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.377726 2575 scope.go:122] "RemoveContainer" containerID="37e3534ab94e2e9ef40efb03a621dbb8b1cef1ca927588c7a9ebdd7f511fdbf6" Oct 29 23:12:47.377798 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.377770 2575 scope.go:122] "RemoveContainer" containerID="a32678eaae6ae6d7b889b7350cba11dc4d854c7dae6d8bfaa12ca78621468dbb" Oct 29 23:12:47.378011 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:47.377991 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2gzh7_openshift-console-operator(947dc777-7dd0-4244-913a-5dd068e6e6a3)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" podUID="947dc777-7dd0-4244-913a-5dd068e6e6a3" Oct 29 23:12:47.379669 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.379638 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dn96d" event={"ID":"b368f5ac-f18a-49d2-b9be-6cca37b7505b","Type":"ContainerStarted","Data":"e008b9fb8b4d2bb09c28afa47ec76dad17bd4525da1dc9650d2142c524970493"} Oct 29 23:12:47.445752 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.445609 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:47.467115 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.467081 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-j6lj7"] Oct 29 23:12:47.498249 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.498215 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-75lxf"] Oct 29 23:12:47.499307 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.499282 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j6lj7" Oct 29 23:12:47.517210 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.516064 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-j6lj7"] Oct 29 23:12:47.517210 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.516179 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-75lxf" Oct 29 23:12:47.517210 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.516584 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 29 23:12:47.517210 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.516637 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vktqh\"" Oct 29 23:12:47.517210 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.516729 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 29 23:12:47.521737 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.521699 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-75lxf"] Oct 29 23:12:47.533015 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.530194 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-hhksv-vlwlb"] Oct 29 23:12:47.544595 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:47.541400 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfce92e88_114c_49c7_b658_193d508126e9.slice/crio-9c3adfcc48550f5cacb81cf8eb7f1bdd2f3ef037fb5cd65766debb6790819d43 WatchSource:0}: Error finding container 9c3adfcc48550f5cacb81cf8eb7f1bdd2f3ef037fb5cd65766debb6790819d43: Status 404 returned error can't find the container with id 9c3adfcc48550f5cacb81cf8eb7f1bdd2f3ef037fb5cd65766debb6790819d43 Oct 29 23:12:47.587436 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.587401 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:47.587569 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.587477 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:47.587569 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:47.587548 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 29 23:12:47.587668 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:47.587590 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm podName:b3b53bf9-0305-43ad-99b1-26f35c841020 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:48.587577609 +0000 UTC m=+37.126301015 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x975f" (UID: "b3b53bf9-0305-43ad-99b1-26f35c841020") : configmap "kube-rbac-proxy" not found Oct 29 23:12:47.587899 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:47.587882 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 29 23:12:47.587968 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:47.587923 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls podName:b3b53bf9-0305-43ad-99b1-26f35c841020 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:48.587912428 +0000 UTC m=+37.126635847 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x975f" (UID: "b3b53bf9-0305-43ad-99b1-26f35c841020") : secret "insights-runtime-extractor-tls" not found Oct 29 23:12:47.608671 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.608594 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:47.653681 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.652271 2575 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-t9kdp"] Oct 29 23:12:47.671463 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.670092 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" Oct 29 23:12:47.675890 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.675717 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-t9kdp"] Oct 29 23:12:47.687637 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.687611 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Oct 29 23:12:47.689487 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.688885 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Oct 29 23:12:47.689487 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.688978 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Oct 29 23:12:47.689487 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.689026 2575 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Oct 29 23:12:47.689487 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.689441 2575 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-b5rdd\"" Oct 29 23:12:47.689487 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.689439 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b24kn\" (UniqueName: \"kubernetes.io/projected/07425f46-dc9b-4d82-a1b8-246162054ea9-kube-api-access-b24kn\") pod \"migrator-66df5d496b-j6lj7\" (UID: \"07425f46-dc9b-4d82-a1b8-246162054ea9\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j6lj7" Oct 29 23:12:47.689703 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.689581 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc28v\" (UniqueName: \"kubernetes.io/projected/7153f183-6e65-4aed-b021-d0a8c3070fce-kube-api-access-hc28v\") pod \"migrator-66df5d496b-75lxf\" (UID: \"7153f183-6e65-4aed-b021-d0a8c3070fce\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-75lxf" Oct 29 23:12:47.766025 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.765928 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:47.790718 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.790683 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chhmv\" (UniqueName: \"kubernetes.io/projected/31ddd1aa-9d35-47bd-9553-9538f3517707-kube-api-access-chhmv\") pod \"service-ca-5fffc97bdf-t9kdp\" (UID: \"31ddd1aa-9d35-47bd-9553-9538f3517707\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" Oct 29 23:12:47.790890 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.790741 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b24kn\" (UniqueName: \"kubernetes.io/projected/07425f46-dc9b-4d82-a1b8-246162054ea9-kube-api-access-b24kn\") pod \"migrator-66df5d496b-j6lj7\" (UID: \"07425f46-dc9b-4d82-a1b8-246162054ea9\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j6lj7" Oct 29 23:12:47.790890 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.790859 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc28v\" (UniqueName: \"kubernetes.io/projected/7153f183-6e65-4aed-b021-d0a8c3070fce-kube-api-access-hc28v\") pod \"migrator-66df5d496b-75lxf\" (UID: \"7153f183-6e65-4aed-b021-d0a8c3070fce\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-75lxf" Oct 29 23:12:47.790890 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.790888 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/31ddd1aa-9d35-47bd-9553-9538f3517707-signing-key\") pod \"service-ca-5fffc97bdf-t9kdp\" (UID: \"31ddd1aa-9d35-47bd-9553-9538f3517707\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" Oct 29 23:12:47.790990 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.790939 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/31ddd1aa-9d35-47bd-9553-9538f3517707-signing-cabundle\") pod \"service-ca-5fffc97bdf-t9kdp\" (UID: \"31ddd1aa-9d35-47bd-9553-9538f3517707\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" Oct 29 23:12:47.816882 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.816849 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc28v\" (UniqueName: \"kubernetes.io/projected/7153f183-6e65-4aed-b021-d0a8c3070fce-kube-api-access-hc28v\") pod \"migrator-66df5d496b-75lxf\" (UID: \"7153f183-6e65-4aed-b021-d0a8c3070fce\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-75lxf" Oct 29 23:12:47.817529 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.817478 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b24kn\" (UniqueName: \"kubernetes.io/projected/07425f46-dc9b-4d82-a1b8-246162054ea9-kube-api-access-b24kn\") pod \"migrator-66df5d496b-j6lj7\" (UID: \"07425f46-dc9b-4d82-a1b8-246162054ea9\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j6lj7" Oct 29 23:12:47.842121 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.842085 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j6lj7" Oct 29 23:12:47.862772 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.862713 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-75lxf" Oct 29 23:12:47.893033 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.891931 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/31ddd1aa-9d35-47bd-9553-9538f3517707-signing-key\") pod \"service-ca-5fffc97bdf-t9kdp\" (UID: \"31ddd1aa-9d35-47bd-9553-9538f3517707\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" Oct 29 23:12:47.893033 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.892048 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/31ddd1aa-9d35-47bd-9553-9538f3517707-signing-cabundle\") pod \"service-ca-5fffc97bdf-t9kdp\" (UID: \"31ddd1aa-9d35-47bd-9553-9538f3517707\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" Oct 29 23:12:47.893033 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.892182 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chhmv\" (UniqueName: \"kubernetes.io/projected/31ddd1aa-9d35-47bd-9553-9538f3517707-kube-api-access-chhmv\") pod \"service-ca-5fffc97bdf-t9kdp\" (UID: \"31ddd1aa-9d35-47bd-9553-9538f3517707\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" Oct 29 23:12:47.894088 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.894054 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/31ddd1aa-9d35-47bd-9553-9538f3517707-signing-cabundle\") pod \"service-ca-5fffc97bdf-t9kdp\" (UID: \"31ddd1aa-9d35-47bd-9553-9538f3517707\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" Oct 29 23:12:47.897103 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.897056 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/31ddd1aa-9d35-47bd-9553-9538f3517707-signing-key\") pod \"service-ca-5fffc97bdf-t9kdp\" (UID: \"31ddd1aa-9d35-47bd-9553-9538f3517707\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" Oct 29 23:12:47.918006 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:47.917945 2575 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chhmv\" (UniqueName: \"kubernetes.io/projected/31ddd1aa-9d35-47bd-9553-9538f3517707-kube-api-access-chhmv\") pod \"service-ca-5fffc97bdf-t9kdp\" (UID: \"31ddd1aa-9d35-47bd-9553-9538f3517707\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" Oct 29 23:12:48.010855 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.010620 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" Oct 29 23:12:48.064673 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.064584 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:48.173741 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.173715 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-j6lj7"] Oct 29 23:12:48.176800 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:48.176768 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07425f46_dc9b_4d82_a1b8_246162054ea9.slice/crio-382bd5f104a5df266b157f80d862e46f2a5dc5db186f38c34a273b90b216499e WatchSource:0}: Error finding container 382bd5f104a5df266b157f80d862e46f2a5dc5db186f38c34a273b90b216499e: Status 404 returned error can't find the container with id 382bd5f104a5df266b157f80d862e46f2a5dc5db186f38c34a273b90b216499e Oct 29 23:12:48.208483 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.208458 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-75lxf"] Oct 29 23:12:48.236770 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.236738 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:48.267112 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:48.267079 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7153f183_6e65_4aed_b021_d0a8c3070fce.slice/crio-2656c9a13e8e7c08726889a49491f091ce0278df09e873d2e1ba0bb4b77c81fc WatchSource:0}: Error finding container 2656c9a13e8e7c08726889a49491f091ce0278df09e873d2e1ba0bb4b77c81fc: Status 404 returned error can't find the container with id 2656c9a13e8e7c08726889a49491f091ce0278df09e873d2e1ba0bb4b77c81fc Oct 29 23:12:48.370323 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.370298 2575 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-t9kdp"] Oct 29 23:12:48.373140 ip-10-0-130-86 kubenswrapper[2575]: W1029 23:12:48.373109 2575 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31ddd1aa_9d35_47bd_9553_9538f3517707.slice/crio-c6475c759ad10fe4752007d759540cf86ea6626d354b9e60a62e848556883ae2 WatchSource:0}: Error finding container c6475c759ad10fe4752007d759540cf86ea6626d354b9e60a62e848556883ae2: Status 404 returned error can't find the container with id c6475c759ad10fe4752007d759540cf86ea6626d354b9e60a62e848556883ae2 Oct 29 23:12:48.384536 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.384507 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-hhksv-vlwlb" event={"ID":"fce92e88-114c-49c7-b658-193d508126e9","Type":"ContainerStarted","Data":"a17b68680b921ac2171d902a12531b30bbec6f97957f90c0a88b4a9c5d68da08"} Oct 29 23:12:48.384641 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.384567 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-hhksv-vlwlb" event={"ID":"fce92e88-114c-49c7-b658-193d508126e9","Type":"ContainerStarted","Data":"9c3adfcc48550f5cacb81cf8eb7f1bdd2f3ef037fb5cd65766debb6790819d43"} Oct 29 23:12:48.385935 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.385905 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-75lxf" event={"ID":"7153f183-6e65-4aed-b021-d0a8c3070fce","Type":"ContainerStarted","Data":"2656c9a13e8e7c08726889a49491f091ce0278df09e873d2e1ba0bb4b77c81fc"} Oct 29 23:12:48.389354 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.389333 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/1.log" Oct 29 23:12:48.389717 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.389697 2575 scope.go:122] "RemoveContainer" containerID="a32678eaae6ae6d7b889b7350cba11dc4d854c7dae6d8bfaa12ca78621468dbb" Oct 29 23:12:48.389902 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:48.389880 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2gzh7_openshift-console-operator(947dc777-7dd0-4244-913a-5dd068e6e6a3)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" podUID="947dc777-7dd0-4244-913a-5dd068e6e6a3" Oct 29 23:12:48.391327 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.391301 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" event={"ID":"31ddd1aa-9d35-47bd-9553-9538f3517707","Type":"ContainerStarted","Data":"c6475c759ad10fe4752007d759540cf86ea6626d354b9e60a62e848556883ae2"} Oct 29 23:12:48.393356 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.393332 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j6lj7" event={"ID":"07425f46-dc9b-4d82-a1b8-246162054ea9","Type":"ContainerStarted","Data":"382bd5f104a5df266b157f80d862e46f2a5dc5db186f38c34a273b90b216499e"} Oct 29 23:12:48.395134 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.395059 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t8vj4" event={"ID":"fa19a031-f171-495b-a72d-f4f79b88fceb","Type":"ContainerStarted","Data":"cf56308124f9f7679660c9fe2722aa01e06d36820dfe0b817abf97ae68e8d748"} Oct 29 23:12:48.478405 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.478341 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-t8vj4" podStartSLOduration=2.184943258 podStartE2EDuration="7.478322052s" totalImagesPullingTime="5.293378794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:12:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:48.469618445 +0000 UTC m=+37.008341875" watchObservedRunningTime="2026-10-29 23:12:48.478322052 +0000 UTC m=+37.017045481" Oct 29 23:12:48.479359 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.479320 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-hhksv-vlwlb" podStartSLOduration=2.479308132 podStartE2EDuration="2.479308132s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:12:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:48.428097561 +0000 UTC m=+36.966820991" watchObservedRunningTime="2026-10-29 23:12:48.479308132 +0000 UTC m=+37.018031561" Oct 29 23:12:48.599816 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.599772 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:48.600010 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.599909 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:48.601033 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:48.600076 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 29 23:12:48.601033 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:48.600144 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm podName:b3b53bf9-0305-43ad-99b1-26f35c841020 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:50.600124116 +0000 UTC m=+39.138847525 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x975f" (UID: "b3b53bf9-0305-43ad-99b1-26f35c841020") : configmap "kube-rbac-proxy" not found Oct 29 23:12:48.601033 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:48.600575 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 29 23:12:48.601033 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:48.600626 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls podName:b3b53bf9-0305-43ad-99b1-26f35c841020 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:50.60061181 +0000 UTC m=+39.139335219 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x975f" (UID: "b3b53bf9-0305-43ad-99b1-26f35c841020") : secret "insights-runtime-extractor-tls" not found Oct 29 23:12:48.848820 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:48.848621 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:49.107249 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:49.106503 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:49.107249 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:49.106599 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:49.107249 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:49.106648 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:49.107249 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:49.106943 2575 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 29 23:12:49.107249 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:49.106961 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle podName:b6ad3759-e11c-476d-82ee-fffdf8fd6d47 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.106942325 +0000 UTC m=+45.645665746 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle") pod "router-default-866787b6f9-24krt" (UID: "b6ad3759-e11c-476d-82ee-fffdf8fd6d47") : configmap references non-existent config key: service-ca.crt Oct 29 23:12:49.107249 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:49.107003 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls podName:63784b03-969b-4e2a-b799-e027ad1c0183 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.106991155 +0000 UTC m=+45.645714564 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-t6gcj" (UID: "63784b03-969b-4e2a-b799-e027ad1c0183") : secret "cluster-monitoring-operator-tls" not found Oct 29 23:12:49.107249 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:49.107131 2575 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 29 23:12:49.107249 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:49.107166 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs podName:b6ad3759-e11c-476d-82ee-fffdf8fd6d47 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.107154876 +0000 UTC m=+45.645878297 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs") pod "router-default-866787b6f9-24krt" (UID: "b6ad3759-e11c-476d-82ee-fffdf8fd6d47") : secret "router-metrics-certs-default" not found Oct 29 23:12:49.210098 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:49.208833 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:49.210098 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:49.208939 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-b9jng\" (UID: \"b218e5d9-83c0-4976-90da-471e6b86d460\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" Oct 29 23:12:49.210098 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:49.208989 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:49.210098 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:49.209029 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:49.210098 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:49.209076 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:49.210098 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:49.209250 2575 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 29 23:12:49.210098 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:49.209330 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls podName:8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.20930916 +0000 UTC m=+45.748032580 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-jfvvm" (UID: "8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4") : secret "samples-operator-tls" not found Oct 29 23:12:49.210098 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:49.209733 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 23:12:49.210098 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:49.209757 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5679dbd569-548pj: secret "image-registry-tls" not found Oct 29 23:12:49.210098 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:49.209800 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls podName:6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.20978501 +0000 UTC m=+45.748508421 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls") pod "image-registry-5679dbd569-548pj" (UID: "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c") : secret "image-registry-tls" not found Oct 29 23:12:49.210098 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:49.209860 2575 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 29 23:12:49.210098 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:49.209893 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert podName:b218e5d9-83c0-4976-90da-471e6b86d460 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.209882133 +0000 UTC m=+45.748605543 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-b9jng" (UID: "b218e5d9-83c0-4976-90da-471e6b86d460") : secret "networking-console-plugin-cert" not found Oct 29 23:12:49.210098 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:49.209944 2575 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 29 23:12:49.210098 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:49.209974 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls podName:8388282a-8b2c-4f5c-ab3d-ec58ea20cc93 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.209964518 +0000 UTC m=+45.748687943 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls") pod "dns-default-gnwhl" (UID: "8388282a-8b2c-4f5c-ab3d-ec58ea20cc93") : secret "dns-default-metrics-tls" not found Oct 29 23:12:49.210098 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:49.210027 2575 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 29 23:12:49.210098 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:49.210038 2575 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cbdc4b876-9xjlm: secret "image-registry-tls" not found Oct 29 23:12:49.211018 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:49.210065 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls podName:25afb96c-6351-470d-8dfa-dd8a9145be40 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.210055621 +0000 UTC m=+45.748779042 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls") pod "image-registry-7cbdc4b876-9xjlm" (UID: "25afb96c-6351-470d-8dfa-dd8a9145be40") : secret "image-registry-tls" not found Oct 29 23:12:49.406350 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:49.406319 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" event={"ID":"31ddd1aa-9d35-47bd-9553-9538f3517707","Type":"ContainerStarted","Data":"65346b8e8a7f4a214063db1a07a1a7ece43a6cd52bd82370f27954008d7b68f6"} Oct 29 23:12:49.409787 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:49.409405 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nhgxp" event={"ID":"1575b8c7-d3a3-4956-bb56-72ba26dae4f6","Type":"ContainerStarted","Data":"85ce38019c118da54cf9ded12f67824651ba2d6b402dc457be15b7ab3ecec082"} Oct 29 23:12:49.455931 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:49.455682 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" podStartSLOduration=2.455662275 podStartE2EDuration="2.455662275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:12:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:49.454592564 +0000 UTC m=+37.993315993" watchObservedRunningTime="2026-10-29 23:12:49.455662275 +0000 UTC m=+37.994385708" Oct 29 23:12:49.472342 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:49.472282 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:49.501399 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:49.500801 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nhgxp" podStartSLOduration=2052.347828879 podStartE2EDuration="34m18.500779838s" totalImagesPullingTime="6.152950959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 22:38:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:49.499030878 +0000 UTC m=+38.037754308" watchObservedRunningTime="2026-10-29 23:12:49.500779838 +0000 UTC m=+38.039503270" Oct 29 23:12:49.631403 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:49.631309 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:49.783637 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:49.783575 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:49.949978 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:49.949939 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:50.119018 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:50.118987 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:50.147598 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:50.147574 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zr8dl_1211641d-1d2c-4de0-b993-5f1e84edc09f/dns-node-resolver/0.log" Oct 29 23:12:50.414367 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:50.414331 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j6lj7" event={"ID":"07425f46-dc9b-4d82-a1b8-246162054ea9","Type":"ContainerStarted","Data":"c161b8f22be237ea841dad002ac9310cef3ee91b34010a07990a5a93d68b6cc2"} Oct 29 23:12:50.416275 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:50.416241 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dn96d" event={"ID":"b368f5ac-f18a-49d2-b9be-6cca37b7505b","Type":"ContainerStarted","Data":"89c3170f2079ee84bcbfb6a5d5eb8d71c764b5e86e97abd41b7d0b2587067d8b"} Oct 29 23:12:50.416405 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:50.416327 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:50.418731 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:50.418708 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-75lxf" event={"ID":"7153f183-6e65-4aed-b021-d0a8c3070fce","Type":"ContainerStarted","Data":"99ac6033f5f36c8c6e472c68f4a641e3e5140a26f9e88d5be4c7e4c600d3f688"} Oct 29 23:12:50.461132 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:50.461087 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j6lj7" podStartSLOduration=1.5974476499999999 podStartE2EDuration="3.461069815s" totalImagesPullingTime="1.863622165s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:12:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:50.459512178 +0000 UTC m=+38.998235606" watchObservedRunningTime="2026-10-29 23:12:50.461069815 +0000 UTC m=+38.999793240" Oct 29 23:12:50.504136 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:50.504091 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dn96d" podStartSLOduration=35.553591056 podStartE2EDuration="38.50407918s" totalImagesPullingTime="2.950488124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:50.503344698 +0000 UTC m=+39.042068127" watchObservedRunningTime="2026-10-29 23:12:50.50407918 +0000 UTC m=+39.042802670" Oct 29 23:12:50.547892 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:50.547833 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-75lxf" podStartSLOduration=1.535320178 podStartE2EDuration="3.547815746s" totalImagesPullingTime="2.012495568s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:12:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:50.547295817 +0000 UTC m=+39.086019247" watchObservedRunningTime="2026-10-29 23:12:50.547815746 +0000 UTC m=+39.086539173" Oct 29 23:12:50.625708 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:50.625622 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:50.625881 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:50.625711 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:50.625881 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:50.625781 2575 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 29 23:12:50.625881 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:50.625784 2575 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 29 23:12:50.625881 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:50.625850 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm podName:b3b53bf9-0305-43ad-99b1-26f35c841020 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:54.625834082 +0000 UTC m=+43.164557510 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x975f" (UID: "b3b53bf9-0305-43ad-99b1-26f35c841020") : configmap "kube-rbac-proxy" not found Oct 29 23:12:50.625881 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:50.625864 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls podName:b3b53bf9-0305-43ad-99b1-26f35c841020 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:54.625858075 +0000 UTC m=+43.164581482 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x975f" (UID: "b3b53bf9-0305-43ad-99b1-26f35c841020") : secret "insights-runtime-extractor-tls" not found Oct 29 23:12:50.939022 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:50.938995 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jg249_c652cee1-24f4-460a-8e1a-669b0ae9c0b1/node-ca/0.log" Oct 29 23:12:51.001334 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:51.001294 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:51.163862 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:51.163825 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:51.357061 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:51.356977 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:51.564490 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:51.564451 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:51.720424 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:51.720393 2575 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:51.720586 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:51.720480 2575 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:51.720888 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:51.720868 2575 scope.go:122] "RemoveContainer" containerID="a32678eaae6ae6d7b889b7350cba11dc4d854c7dae6d8bfaa12ca78621468dbb" Oct 29 23:12:51.721127 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:51.721091 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2gzh7_openshift-console-operator(947dc777-7dd0-4244-913a-5dd068e6e6a3)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" podUID="947dc777-7dd0-4244-913a-5dd068e6e6a3" Oct 29 23:12:51.739037 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:51.739014 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t8vj4_fa19a031-f171-495b-a72d-f4f79b88fceb/serve-healthcheck-canary/0.log" Oct 29 23:12:51.764387 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:51.764361 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:51.941642 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:51.941615 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-75lxf_7153f183-6e65-4aed-b021-d0a8c3070fce/migrator/0.log" Oct 29 23:12:51.974100 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:51.974017 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:52.129212 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:52.129181 2575 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:52.138690 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:52.138671 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-j6lj7_07425f46-dc9b-4d82-a1b8-246162054ea9/migrator/0.log" Oct 29 23:12:52.340063 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:52.340042 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-mc79c_8f11a401-ced0-47dc-8814-c23849ba794b/kube-storage-version-migrator-operator/0.log" Oct 29 23:12:52.427391 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:52.427354 2575 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gz4mg" event={"ID":"cf41b924-24b7-4a89-9d11-af8b4246fcde","Type":"ContainerStarted","Data":"fd531c7268da9b048a5caa6cf71044a9bb787a7f75b9362ceb2c30dd8e1a18f5"} Oct 29 23:12:52.427751 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:52.427733 2575 scope.go:122] "RemoveContainer" containerID="a32678eaae6ae6d7b889b7350cba11dc4d854c7dae6d8bfaa12ca78621468dbb" Oct 29 23:12:52.427962 ip-10-0-130-86 kubenswrapper[2575]: E1029 23:12:52.427940 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2gzh7_openshift-console-operator(947dc777-7dd0-4244-913a-5dd068e6e6a3)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" podUID="947dc777-7dd0-4244-913a-5dd068e6e6a3" Oct 29 23:12:52.473211 ip-10-0-130-86 kubenswrapper[2575]: I1029 23:12:52.473164 2575 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gz4mg" podStartSLOduration=34.386365897 podStartE2EDuration="40.47315133s" totalImagesPullingTime="6.086785433s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:12:52.473061476 +0000 UTC m=+41.011784905" watchObservedRunningTime="2026-10-29 23:12:52.47315133 +0000 UTC m=+41.011874757" Oct 29 23:12:52.554523 ip-10-0-130-86 systemd[1]: Stopping Kubernetes Kubelet... Oct 29 23:12:52.574639 ip-10-0-130-86 systemd[1]: kubelet.service: Deactivated successfully. Oct 29 23:12:52.574808 ip-10-0-130-86 systemd[1]: Stopped Kubernetes Kubelet. Oct 29 23:12:52.574856 ip-10-0-130-86 systemd[1]: kubelet.service: Consumed 3.251s CPU time. Oct 29 23:12:52.589915 ip-10-0-130-86 systemd[1]: Starting Kubernetes Kubelet... Oct 29 23:12:52.680177 ip-10-0-130-86 kubenswrapper[5056]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 29 23:12:52.680177 ip-10-0-130-86 kubenswrapper[5056]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 29 23:12:52.680177 ip-10-0-130-86 kubenswrapper[5056]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 29 23:12:52.680177 ip-10-0-130-86 kubenswrapper[5056]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 29 23:12:52.682241 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682153 5056 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 23:12:52.682241 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682242 5056 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682247 5056 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682250 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682253 5056 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682270 5056 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682275 5056 feature_gate.go:386] unrecognized feature gate: Example Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682278 5056 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682281 5056 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682284 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682286 5056 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682289 5056 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682292 5056 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682294 5056 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682297 5056 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682300 5056 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682304 5056 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682306 5056 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682309 5056 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682312 5056 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682314 5056 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 23:12:52.682321 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682317 5056 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682319 5056 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682323 5056 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682325 5056 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682328 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682337 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682340 5056 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682343 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682346 5056 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682348 5056 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682351 5056 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682354 5056 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682356 5056 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682359 5056 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682362 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682365 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682367 5056 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682370 5056 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682372 5056 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 23:12:52.682903 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682375 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682378 5056 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682380 5056 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682383 5056 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682385 5056 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682388 5056 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682391 5056 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682393 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682396 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682398 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682407 5056 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682409 5056 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682412 5056 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682416 5056 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682419 5056 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682421 5056 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682424 5056 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682427 5056 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682429 5056 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 23:12:52.683403 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682432 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682436 5056 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682440 5056 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682443 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682446 5056 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682450 5056 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682453 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682457 5056 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682460 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682463 5056 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682466 5056 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682469 5056 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682472 5056 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682474 5056 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682477 5056 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682480 5056 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682482 5056 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682485 5056 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682488 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 23:12:52.683898 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682491 5056 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682493 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682496 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682499 5056 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682503 5056 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682506 5056 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682509 5056 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682512 5056 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682515 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682518 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682520 5056 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682523 5056 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682525 5056 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682528 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682530 5056 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682533 5056 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682535 5056 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682538 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682541 5056 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682543 5056 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 23:12:52.684421 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682546 5056 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682548 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682551 5056 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682553 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682556 5056 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682560 5056 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682563 5056 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.682565 5056 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683332 5056 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683340 5056 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683343 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683346 5056 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683349 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683352 5056 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683354 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683357 5056 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683361 5056 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683363 5056 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683366 5056 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683369 5056 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 23:12:52.684910 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683379 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683382 5056 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683385 5056 feature_gate.go:386] unrecognized feature gate: Example Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683388 5056 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683390 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683393 5056 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683396 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683398 5056 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683401 5056 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683403 5056 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683406 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683409 5056 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683412 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683414 5056 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683417 5056 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683420 5056 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683423 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683426 5056 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683428 5056 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683431 5056 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 23:12:52.685447 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683434 5056 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683436 5056 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683439 5056 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683442 5056 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683444 5056 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683447 5056 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683449 5056 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683452 5056 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683454 5056 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683457 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683460 5056 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683464 5056 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683468 5056 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683476 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683480 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683483 5056 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683486 5056 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683488 5056 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683491 5056 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 23:12:52.685956 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683494 5056 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683496 5056 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683499 5056 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683502 5056 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683505 5056 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683508 5056 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683511 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683514 5056 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683516 5056 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683519 5056 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683521 5056 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683526 5056 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683530 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683533 5056 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683536 5056 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683538 5056 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683541 5056 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683543 5056 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683546 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 23:12:52.686454 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683549 5056 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683552 5056 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683554 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683557 5056 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683560 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683563 5056 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683566 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683568 5056 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683576 5056 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683579 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683584 5056 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683587 5056 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683590 5056 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683592 5056 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683595 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683604 5056 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683606 5056 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683609 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683612 5056 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683615 5056 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 23:12:52.686918 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683617 5056 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683620 5056 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683623 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683625 5056 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683628 5056 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683631 5056 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683633 5056 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683636 5056 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683638 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683641 5056 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683643 5056 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683646 5056 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683648 5056 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683650 5056 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683653 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.683656 5056 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684195 5056 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684201 5056 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684203 5056 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684207 5056 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 23:12:52.687414 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684215 5056 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684218 5056 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684223 5056 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684226 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684228 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684231 5056 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684234 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684237 5056 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684239 5056 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684242 5056 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684244 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684247 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684250 5056 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684252 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684254 5056 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684272 5056 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684275 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684278 5056 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684280 5056 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 23:12:52.687917 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684283 5056 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684286 5056 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684289 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684291 5056 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684294 5056 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684296 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684299 5056 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684302 5056 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684305 5056 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684307 5056 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684310 5056 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684312 5056 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684315 5056 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684318 5056 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684328 5056 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684333 5056 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684336 5056 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684339 5056 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684341 5056 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684344 5056 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 23:12:52.688402 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684346 5056 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684350 5056 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684354 5056 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684357 5056 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684360 5056 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684362 5056 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684365 5056 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684368 5056 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684370 5056 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684373 5056 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684376 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684378 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684381 5056 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684384 5056 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684386 5056 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684389 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684392 5056 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684395 5056 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684397 5056 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 23:12:52.688893 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684400 5056 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684402 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684405 5056 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684407 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684410 5056 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684413 5056 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684415 5056 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684418 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684427 5056 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684431 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684434 5056 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684436 5056 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684440 5056 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684442 5056 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684445 5056 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684447 5056 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684450 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684453 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684455 5056 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 23:12:52.689671 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684458 5056 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684462 5056 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684465 5056 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684467 5056 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684470 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684473 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684475 5056 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684478 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684480 5056 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684484 5056 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684487 5056 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684490 5056 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684492 5056 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684495 5056 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684497 5056 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684499 5056 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684502 5056 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684504 5056 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684507 5056 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 23:12:52.690242 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684509 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684512 5056 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684516 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684524 5056 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684527 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.684530 5056 feature_gate.go:386] unrecognized feature gate: Example Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684651 5056 flags.go:64] FLAG: --address="0.0.0.0" Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684664 5056 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684673 5056 flags.go:64] FLAG: --anonymous-auth="true" Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684678 5056 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684682 5056 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684686 5056 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684690 5056 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684694 5056 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684704 5056 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684707 5056 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684711 5056 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684714 5056 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684718 5056 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684721 5056 flags.go:64] FLAG: --cgroup-root="" Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684724 5056 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684727 5056 flags.go:64] FLAG: --client-ca-file="" Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684730 5056 flags.go:64] FLAG: --cloud-provider="external" Oct 29 23:12:52.690737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684733 5056 flags.go:64] FLAG: --cluster-dns="[]" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684743 5056 flags.go:64] FLAG: --cluster-domain="" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684746 5056 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684749 5056 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684752 5056 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684756 5056 flags.go:64] FLAG: --container-log-max-files="5" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684760 5056 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684763 5056 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684766 5056 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684770 5056 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684773 5056 flags.go:64] FLAG: --contention-profiling="false" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684777 5056 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684781 5056 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684795 5056 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684799 5056 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684803 5056 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684806 5056 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684809 5056 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684813 5056 flags.go:64] FLAG: --enable-load-reader="false" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684816 5056 flags.go:64] FLAG: --enable-server="true" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684819 5056 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684826 5056 flags.go:64] FLAG: --event-burst="100" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684829 5056 flags.go:64] FLAG: --event-qps="50" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684832 5056 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 29 23:12:52.691306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684835 5056 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684838 5056 flags.go:64] FLAG: --eviction-hard="" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684842 5056 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684845 5056 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684849 5056 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684852 5056 flags.go:64] FLAG: --eviction-soft="" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684855 5056 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684858 5056 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684861 5056 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684864 5056 flags.go:64] FLAG: --experimental-mounter-path="" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684868 5056 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684871 5056 flags.go:64] FLAG: --fail-swap-on="true" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684874 5056 flags.go:64] FLAG: --feature-gates="" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684878 5056 flags.go:64] FLAG: --file-check-frequency="20s" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684881 5056 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684885 5056 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684888 5056 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684891 5056 flags.go:64] FLAG: --healthz-port="10248" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684894 5056 flags.go:64] FLAG: --help="false" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684899 5056 flags.go:64] FLAG: --hostname-override="ip-10-0-130-86.ec2.internal" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684902 5056 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684906 5056 flags.go:64] FLAG: --http-check-frequency="20s" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684914 5056 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684918 5056 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 29 23:12:52.691946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684921 5056 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684925 5056 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684928 5056 flags.go:64] FLAG: --image-service-endpoint="" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684931 5056 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684934 5056 flags.go:64] FLAG: --kube-api-burst="100" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684937 5056 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684940 5056 flags.go:64] FLAG: --kube-api-qps="50" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684943 5056 flags.go:64] FLAG: --kube-reserved="" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684946 5056 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684949 5056 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684952 5056 flags.go:64] FLAG: --kubelet-cgroups="" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684955 5056 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684958 5056 flags.go:64] FLAG: --lock-file="" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684961 5056 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684964 5056 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684967 5056 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684973 5056 flags.go:64] FLAG: --log-json-split-stream="false" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684976 5056 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684979 5056 flags.go:64] FLAG: --log-text-split-stream="false" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684982 5056 flags.go:64] FLAG: --logging-format="text" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684985 5056 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684989 5056 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684992 5056 flags.go:64] FLAG: --manifest-url="" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.684995 5056 flags.go:64] FLAG: --manifest-url-header="" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685000 5056 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 29 23:12:52.692553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685003 5056 flags.go:64] FLAG: --max-open-files="1000000" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685007 5056 flags.go:64] FLAG: --max-pods="110" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685011 5056 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685014 5056 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685017 5056 flags.go:64] FLAG: --memory-manager-policy="None" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685020 5056 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685029 5056 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685033 5056 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685036 5056 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685043 5056 flags.go:64] FLAG: --node-status-max-images="50" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685046 5056 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685049 5056 flags.go:64] FLAG: --oom-score-adj="-999" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685052 5056 flags.go:64] FLAG: --pod-cidr="" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685055 5056 flags.go:64] FLAG: --pod-manifest-path="" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685058 5056 flags.go:64] FLAG: --pod-max-pids="-1" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685061 5056 flags.go:64] FLAG: --pods-per-core="0" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685064 5056 flags.go:64] FLAG: --port="10250" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685067 5056 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685070 5056 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-056125f30a3a4f1a1" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685073 5056 flags.go:64] FLAG: --qos-reserved="" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685078 5056 flags.go:64] FLAG: --read-only-port="10255" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685081 5056 flags.go:64] FLAG: --register-node="true" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685084 5056 flags.go:64] FLAG: --register-with-taints="" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685088 5056 flags.go:64] FLAG: --registry-burst="10" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685091 5056 flags.go:64] FLAG: --registry-qps="5" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685094 5056 flags.go:64] FLAG: --reserved-cpus="" Oct 29 23:12:52.693164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685097 5056 flags.go:64] FLAG: --reserved-memory="" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685101 5056 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685104 5056 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685107 5056 flags.go:64] FLAG: --rotate-certificates="false" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685110 5056 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685113 5056 flags.go:64] FLAG: --runonce="false" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685116 5056 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685119 5056 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685123 5056 flags.go:64] FLAG: --seccomp-default="false" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685126 5056 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685129 5056 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685132 5056 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685136 5056 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685144 5056 flags.go:64] FLAG: --storage-driver-password="root" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685148 5056 flags.go:64] FLAG: --storage-driver-secure="false" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685151 5056 flags.go:64] FLAG: --storage-driver-table="stats" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685154 5056 flags.go:64] FLAG: --storage-driver-user="root" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685157 5056 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685160 5056 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685163 5056 flags.go:64] FLAG: --system-cgroups="" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685166 5056 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685171 5056 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685174 5056 flags.go:64] FLAG: --tls-cert-file="" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685177 5056 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685183 5056 flags.go:64] FLAG: --tls-min-version="" Oct 29 23:12:52.693816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685186 5056 flags.go:64] FLAG: --tls-private-key-file="" Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685190 5056 flags.go:64] FLAG: --topology-manager-policy="none" Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685193 5056 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685196 5056 flags.go:64] FLAG: --topology-manager-scope="container" Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685199 5056 flags.go:64] FLAG: --v="2" Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685203 5056 flags.go:64] FLAG: --version="false" Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685208 5056 flags.go:64] FLAG: --vmodule="" Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685211 5056 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.685215 5056 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685404 5056 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685410 5056 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685414 5056 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685418 5056 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685421 5056 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685424 5056 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685429 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685432 5056 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685435 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685437 5056 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685440 5056 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685443 5056 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 23:12:52.694450 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685452 5056 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685455 5056 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685458 5056 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685460 5056 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685463 5056 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685466 5056 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685468 5056 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685471 5056 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685474 5056 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685476 5056 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685479 5056 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685481 5056 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685486 5056 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685489 5056 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685491 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685494 5056 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685497 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685499 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685502 5056 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 23:12:52.694983 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685505 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685507 5056 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685510 5056 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685513 5056 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685515 5056 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685518 5056 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685520 5056 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685524 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685526 5056 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685529 5056 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685531 5056 feature_gate.go:386] unrecognized feature gate: Example Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685534 5056 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685537 5056 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685539 5056 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685547 5056 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685550 5056 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685553 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685555 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685558 5056 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685560 5056 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 23:12:52.695465 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685563 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685565 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685568 5056 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685570 5056 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685573 5056 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685576 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685579 5056 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685582 5056 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685585 5056 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685587 5056 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685590 5056 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685593 5056 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685595 5056 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685597 5056 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685600 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685602 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685605 5056 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685608 5056 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685610 5056 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685614 5056 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 23:12:52.695976 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685618 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685622 5056 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685625 5056 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685628 5056 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685630 5056 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685633 5056 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685636 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685989 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685992 5056 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685995 5056 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.685997 5056 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686000 5056 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686003 5056 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686005 5056 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686008 5056 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686011 5056 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686013 5056 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686017 5056 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686020 5056 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 23:12:52.696627 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686022 5056 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 23:12:52.697123 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686025 5056 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 23:12:52.697123 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686028 5056 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 23:12:52.697123 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686030 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 23:12:52.697123 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686033 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 23:12:52.697123 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686036 5056 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 23:12:52.697123 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686038 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 23:12:52.697123 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686041 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 23:12:52.697123 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686043 5056 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 23:12:52.697123 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686046 5056 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 23:12:52.697123 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686048 5056 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 23:12:52.697123 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686051 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 23:12:52.697123 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686053 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 23:12:52.697123 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686056 5056 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 23:12:52.697123 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686059 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 23:12:52.697123 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686061 5056 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 23:12:52.697123 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.686089 5056 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686128 5056 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686133 5056 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686136 5056 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686139 5056 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686142 5056 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686145 5056 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686147 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686150 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686152 5056 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686155 5056 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686157 5056 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686162 5056 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686166 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686170 5056 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686173 5056 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686176 5056 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686178 5056 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686181 5056 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 23:12:52.697722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686184 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686186 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686189 5056 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686191 5056 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686194 5056 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686197 5056 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686199 5056 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686202 5056 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686204 5056 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686207 5056 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686210 5056 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686212 5056 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686215 5056 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686217 5056 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686220 5056 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686223 5056 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686225 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686228 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686230 5056 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 23:12:52.698192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686233 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686236 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686238 5056 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686241 5056 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686243 5056 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686246 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686250 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686253 5056 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686271 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686274 5056 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686277 5056 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686281 5056 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686285 5056 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686289 5056 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686292 5056 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686295 5056 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686298 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686300 5056 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686303 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 23:12:52.698736 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686305 5056 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686308 5056 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686310 5056 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686313 5056 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686315 5056 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686318 5056 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686320 5056 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686323 5056 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686326 5056 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686328 5056 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686331 5056 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686333 5056 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686336 5056 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686338 5056 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686341 5056 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686344 5056 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686347 5056 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686349 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686352 5056 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686356 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 23:12:52.699413 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686359 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686363 5056 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686365 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686368 5056 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686371 5056 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686373 5056 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686376 5056 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686379 5056 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686381 5056 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686384 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686387 5056 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686389 5056 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686393 5056 feature_gate.go:386] unrecognized feature gate: Example Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686395 5056 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686398 5056 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686400 5056 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686403 5056 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686405 5056 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686408 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686411 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 23:12:52.700039 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686413 5056 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686416 5056 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686418 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686421 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686423 5056 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686426 5056 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686428 5056 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686431 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686434 5056 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.686436 5056 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.691357 5056 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.691373 5056 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.691394 5056 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.691401 5056 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691466 5056 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691471 5056 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691474 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691477 5056 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691480 5056 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691483 5056 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 23:12:52.700541 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691486 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691488 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691492 5056 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691496 5056 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691500 5056 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691503 5056 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691507 5056 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691509 5056 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691512 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691515 5056 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691517 5056 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691520 5056 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691522 5056 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691525 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691527 5056 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691530 5056 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691538 5056 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691541 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691544 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 23:12:52.701017 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691546 5056 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691549 5056 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691551 5056 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691554 5056 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691556 5056 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691559 5056 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691562 5056 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691565 5056 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691567 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691570 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691572 5056 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691575 5056 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691577 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691580 5056 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691582 5056 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691585 5056 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691588 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691590 5056 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691593 5056 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691596 5056 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 23:12:52.701482 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691598 5056 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691601 5056 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691603 5056 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691606 5056 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691608 5056 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691611 5056 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691613 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691616 5056 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691619 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691621 5056 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691624 5056 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691626 5056 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691629 5056 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691631 5056 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691634 5056 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691636 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691639 5056 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691642 5056 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691644 5056 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691647 5056 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 23:12:52.701965 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691650 5056 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691653 5056 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691656 5056 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691658 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691661 5056 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691664 5056 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691666 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691669 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691671 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691674 5056 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691677 5056 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691679 5056 feature_gate.go:386] unrecognized feature gate: Example Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691682 5056 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691684 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691687 5056 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691690 5056 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691693 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691696 5056 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691698 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 23:12:52.702835 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691701 5056 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691703 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691706 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691708 5056 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691711 5056 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691713 5056 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691717 5056 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691721 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691724 5056 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691727 5056 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691729 5056 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691732 5056 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691735 5056 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691738 5056 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691740 5056 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691743 5056 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691745 5056 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691748 5056 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691750 5056 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 23:12:52.703603 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691753 5056 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 23:12:52.704069 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691755 5056 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 23:12:52.704069 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691758 5056 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 23:12:52.704069 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.691786 5056 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 29 23:12:52.704069 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691941 5056 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 23:12:52.704069 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691947 5056 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 23:12:52.704069 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691950 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 23:12:52.704069 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691953 5056 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 23:12:52.704069 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691956 5056 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 23:12:52.704069 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691962 5056 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 23:12:52.704069 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691965 5056 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 23:12:52.704069 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691969 5056 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 23:12:52.704069 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691972 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 23:12:52.704069 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691975 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 23:12:52.704069 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691977 5056 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 23:12:52.704069 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691980 5056 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691983 5056 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691985 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691988 5056 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691991 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691993 5056 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691996 5056 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.691999 5056 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692001 5056 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692004 5056 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692007 5056 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692010 5056 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692012 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692016 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692018 5056 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692021 5056 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692024 5056 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692026 5056 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692029 5056 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 23:12:52.704460 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692032 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692034 5056 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692037 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692039 5056 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692042 5056 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692045 5056 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692047 5056 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692050 5056 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692053 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692059 5056 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692063 5056 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692067 5056 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692070 5056 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692073 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692076 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692079 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692082 5056 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692084 5056 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692087 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 23:12:52.704944 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692089 5056 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692092 5056 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692095 5056 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692097 5056 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692100 5056 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692103 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692105 5056 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692108 5056 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692110 5056 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692113 5056 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692115 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692118 5056 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692121 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692123 5056 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692126 5056 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692128 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692131 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692133 5056 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692136 5056 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692138 5056 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 23:12:52.705431 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692141 5056 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692144 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692148 5056 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692151 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692153 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692156 5056 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692159 5056 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692162 5056 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692167 5056 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692170 5056 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692172 5056 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692175 5056 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692178 5056 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692181 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692183 5056 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692186 5056 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692188 5056 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692191 5056 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692193 5056 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692196 5056 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 23:12:52.705911 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692199 5056 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 23:12:52.706513 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692201 5056 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 23:12:52.706513 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692204 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 23:12:52.706513 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692206 5056 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 23:12:52.706513 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692209 5056 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 23:12:52.706513 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692212 5056 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 23:12:52.706513 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692214 5056 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 23:12:52.706513 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692217 5056 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 23:12:52.706513 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692219 5056 feature_gate.go:386] unrecognized feature gate: Example Oct 29 23:12:52.706513 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692222 5056 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 23:12:52.706513 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692224 5056 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 23:12:52.706513 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692227 5056 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 23:12:52.706513 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692229 5056 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 23:12:52.706513 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692232 5056 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 23:12:52.706513 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692236 5056 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 23:12:52.706513 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692239 5056 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 23:12:52.706513 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692241 5056 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 23:12:52.706902 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.692281 5056 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 29 23:12:52.706902 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692314 5056 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 29 23:12:52.706902 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692317 5056 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 29 23:12:52.706902 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692321 5056 feature_gate.go:386] unrecognized feature gate: Example Oct 29 23:12:52.706902 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692324 5056 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 29 23:12:52.706902 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692327 5056 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 29 23:12:52.706902 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692329 5056 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 29 23:12:52.706902 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692332 5056 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 29 23:12:52.706902 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692335 5056 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 29 23:12:52.706902 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692338 5056 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 29 23:12:52.706902 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692340 5056 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 29 23:12:52.706902 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692343 5056 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 29 23:12:52.706902 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692346 5056 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 29 23:12:52.706902 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692348 5056 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 29 23:12:52.706902 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692351 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 29 23:12:52.706902 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692354 5056 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692356 5056 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692359 5056 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692362 5056 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692364 5056 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692367 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692369 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692372 5056 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692374 5056 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692377 5056 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692380 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692382 5056 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692385 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692388 5056 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692390 5056 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692394 5056 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692397 5056 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692400 5056 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692403 5056 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 29 23:12:52.707308 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692405 5056 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692410 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692414 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692416 5056 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692419 5056 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692422 5056 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692425 5056 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692427 5056 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692430 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692432 5056 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692435 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692438 5056 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692440 5056 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692443 5056 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692446 5056 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692449 5056 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692452 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692454 5056 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692458 5056 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 29 23:12:52.707771 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692462 5056 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692465 5056 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692467 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692470 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692473 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692475 5056 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692478 5056 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692480 5056 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692483 5056 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692487 5056 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692489 5056 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692492 5056 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692495 5056 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692497 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692501 5056 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692504 5056 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692506 5056 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692509 5056 feature_gate.go:386] unrecognized feature gate: EVPN Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692511 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692514 5056 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 29 23:12:52.708234 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692517 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692520 5056 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692522 5056 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692525 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692528 5056 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692530 5056 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692533 5056 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692536 5056 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692538 5056 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692541 5056 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692543 5056 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692546 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692548 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692551 5056 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692554 5056 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692558 5056 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692561 5056 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692565 5056 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692568 5056 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 29 23:12:52.708740 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692570 5056 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692573 5056 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692577 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692579 5056 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692582 5056 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692585 5056 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692588 5056 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692592 5056 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692594 5056 feature_gate.go:386] unrecognized feature gate: Example2 Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692597 5056 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692599 5056 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692602 5056 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692604 5056 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692608 5056 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:52.692611 5056 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.692663 5056 server.go:957] "Client rotation is on, will bootstrap in background" Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.693658 5056 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.693712 5056 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.693892 5056 server.go:1014] "Starting client certificate rotation" Oct 29 23:12:52.709192 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.694028 5056 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 29 23:12:52.709669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.694071 5056 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-28 23:07:11 +0000 UTC" deadline="2028-07-21 17:26:46.983089248 +0000 UTC" Oct 29 23:12:52.709669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.694131 5056 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15138h13m54.288959899s" Oct 29 23:12:52.709669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.694380 5056 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 29 23:12:52.709669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.695476 5056 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 29 23:12:52.709669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.696791 5056 log.go:25] "Validated CRI v1 runtime API" Oct 29 23:12:52.709669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.698516 5056 log.go:25] "Validated CRI v1 image API" Oct 29 23:12:52.709669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.699318 5056 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 29 23:12:52.709669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.701376 5056 fs.go:135] Filesystem UUIDs: map[2c71c32d-ebcc-40e6-8ad9-5114d692bd30:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f6ddfd90-f75a-4f7c-84b2-7a0b35e091af:/dev/nvme0n1p4] Oct 29 23:12:52.709901 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.701397 5056 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/13ed2c84b37ccdceedcaa24fb57881ee76e650cdf5ba3719e10e76572a6873d8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/13ed2c84b37ccdceedcaa24fb57881ee76e650cdf5ba3719e10e76572a6873d8/userdata/shm major:0 minor:300 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/141e20972e4b1ee800df8a6bdcb140a79bc979449f8cb50efcf252d7f2566176/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/141e20972e4b1ee800df8a6bdcb140a79bc979449f8cb50efcf252d7f2566176/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/15fd926f7c91f01a1e15e40ea2b2d3c9949fad4f8bed6f1cec57a3c827730f16/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/15fd926f7c91f01a1e15e40ea2b2d3c9949fad4f8bed6f1cec57a3c827730f16/userdata/shm major:0 minor:228 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2656c9a13e8e7c08726889a49491f091ce0278df09e873d2e1ba0bb4b77c81fc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2656c9a13e8e7c08726889a49491f091ce0278df09e873d2e1ba0bb4b77c81fc/userdata/shm major:0 minor:343 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/382bd5f104a5df266b157f80d862e46f2a5dc5db186f38c34a273b90b216499e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/382bd5f104a5df266b157f80d862e46f2a5dc5db186f38c34a273b90b216499e/userdata/shm major:0 minor:340 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4a780d6c876c96b2ac229f39a9eb1bc78c74918bd006b1fb2016e24048655c15/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4a780d6c876c96b2ac229f39a9eb1bc78c74918bd006b1fb2016e24048655c15/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/50b667f42d34af62fd160fc85feff19064c5a64317e6ce286a2a267e8bf72c07/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/50b667f42d34af62fd160fc85feff19064c5a64317e6ce286a2a267e8bf72c07/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/579027c84fc7add6a266a6a9736f729b2d15b74fc0c1f2f9317ffdd6d25c5d9d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/579027c84fc7add6a266a6a9736f729b2d15b74fc0c1f2f9317ffdd6d25c5d9d/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/82353c5785c533809f1de6dd9818f418c39cccf47e5cde5e41675d3cd4767a9f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/82353c5785c533809f1de6dd9818f418c39cccf47e5cde5e41675d3cd4767a9f/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/91e59f285c148ed7601fd08e73b9fdd80c84a9a2e14036b88020ad258d7218ea/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/91e59f285c148ed7601fd08e73b9fdd80c84a9a2e14036b88020ad258d7218ea/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/93eba272389667f4872d3f6c71e1d9639fc8144342dfee0a04b23f2d93508d3c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/93eba272389667f4872d3f6c71e1d9639fc8144342dfee0a04b23f2d93508d3c/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9c3adfcc48550f5cacb81cf8eb7f1bdd2f3ef037fb5cd65766debb6790819d43/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9c3adfcc48550f5cacb81cf8eb7f1bdd2f3ef037fb5cd65766debb6790819d43/userdata/shm major:0 minor:322 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a2a8836701e6ddf02d832bc5bd6c5d5964e776520c2c9cec1a5f49d7e98774cd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a2a8836701e6ddf02d832bc5bd6c5d5964e776520c2c9cec1a5f49d7e98774cd/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a8b4722e4e6b2bb95dc0ce5220d860476ec452f37aa982d55f570cc0d55a8c91/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a8b4722e4e6b2bb95dc0ce5220d860476ec452f37aa982d55f570cc0d55a8c91/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/baedcc53bf54f0b96ed8967ff1465e6d9552452fdfc48192e1b26666f1976295/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/baedcc53bf54f0b96ed8967ff1465e6d9552452fdfc48192e1b26666f1976295/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c6475c759ad10fe4752007d759540cf86ea6626d354b9e60a62e848556883ae2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c6475c759ad10fe4752007d759540cf86ea6626d354b9e60a62e848556883ae2/userdata/shm major:0 minor:346 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ce41cdd0cdfb13694d7dcafdb1f3ce7ec1f1caf602a4d21511fdef6c0af294fd/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ce41cdd0cdfb13694d7dcafdb1f3ce7ec1f1caf602a4d21511fdef6c0af294fd/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d94ef0ff71197b3c3c5133f0406808e8cc230c141502ffeca63e12e7de44b305/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d94ef0ff71197b3c3c5133f0406808e8cc230c141502ffeca63e12e7de44b305/userdata/shm major:0 minor:89 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e008b9fb8b4d2bb09c28afa47ec76dad17bd4525da1dc9650d2142c524970493/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e008b9fb8b4d2bb09c28afa47ec76dad17bd4525da1dc9650d2142c524970493/userdata/shm major:0 minor:305 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e2e3cb8499a07a3b027a62e49940d2b429c696b285967e574f469eecea1d3bb2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e2e3cb8499a07a3b027a62e49940d2b429c696b285967e574f469eecea1d3bb2/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e2f99c191ef1a42bf8aee30eaaa041e3a03f7536a34173bd03dc701128e6b7a7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e2f99c191ef1a42bf8aee30eaaa041e3a03f7536a34173bd03dc701128e6b7a7/userdata/shm major:0 minor:243 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e4cbf370e140b1541199154f795361d6b0d7b53f23bc73b3585577e1c9f6fd0e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e4cbf370e140b1541199154f795361d6b0d7b53f23bc73b3585577e1c9f6fd0e/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ee7d783cd1ab14cd5213846bce28f544afd16f712d3405efe87a229a08e0e6c3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ee7d783cd1ab14cd5213846bce28f544afd16f712d3405efe87a229a08e0e6c3/userdata/shm major:0 minor:70 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f4267b48ecd99c04495ab3c0a8971ee0931c33620e4b6c0e69d7338f0365f59a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f4267b48ecd99c04495ab3c0a8971ee0931c33620e4b6c0e69d7338f0365f59a/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fbe63081c629c2739579aaea9f2d9cd911ded08084074e23ea18ea78aac22f30/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fbe63081c629c2739579aaea9f2d9cd911ded08084074e23ea18ea78aac22f30/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ff4dfe98edc9d895bbc191f3efdfe4fd60c8092d4a11d6abf41b31fcaff7ebf5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ff4dfe98edc9d895bbc191f3efdfe4fd60c8092d4a11d6abf41b31fcaff7ebf5/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a/volumes/kubernetes.io~empty-dir/tmp major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a/volumes/kubernetes.io~projected/kube-api-access-qf5x4:{mountpoint:/var/lib/kubelet/pods/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a/volumes/kubernetes.io~projected/kube-api-access-qf5x4 major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/07425f46-dc9b-4d82-a1b8-246162054ea9/volumes/kubernetes.io~projected/kube-api-access-b24kn:{mountpoint:/var/lib/kubelet/pods/07425f46-dc9b-4d82-a1b8-246162054ea9/volumes/kubernetes.io~projected/kube-api-access-b24kn major:0 minor:339 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1211641d-1d2c-4de0-b993-5f1e84edc09f/volumes/kubernetes.io~projected/kube-api-access-gh8rg:{mountpoint:/var/lib/kubelet/pods/1211641d-1d2c-4de0-b993-5f1e84edc09f/volumes/kubernetes.io~projected/kube-api-access-gh8rg major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1575b8c7-d3a3-4956-bb56-72ba26dae4f6/volumes/kubernetes.io~projected/kube-api-access-fnjnb:{mountpoint:/var/lib/kubelet/pods/1575b8c7-d3a3-4956-bb56-72ba26dae4f6/volumes/kubernetes.io~projected/kube-api-access-fnjnb major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/21385963-6be4-4e3f-a996-f48b0c9ab0b1/volumes/kubernetes.io~projected/kube-api-access-jfhfl:{mountpoint:/var/lib/kubelet/pods/21385963-6be4-4e3f-a996-f48b0c9ab0b1/volumes/kubernetes.io~projected/kube-api-access-jfhfl major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/25afb96c-6351-470d-8dfa-dd8a9145be40/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/25afb96c-6351-470d-8dfa-dd8a9145be40/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/25afb96c-6351-470d-8dfa-dd8a9145be40/volumes/kubernetes.io~projected/kube-api-access-bzc9p:{mountpoint:/var/lib/kubelet/pods/25afb96c-6351-470d-8dfa-dd8a9145be40/volumes/kubernetes.io~projected/kube-api-access-bzc9p major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/25afb96c-6351-470d-8dfa-dd8a9145be40/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/25afb96c-6351-470d-8dfa-dd8a9145be40/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/25afb96c-6351-470d-8dfa-dd8a9145be40/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/25afb96c-6351-470d-8dfa-dd8a9145be40/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/31ddd1aa-9d35-47bd-9553-9538f3517707/volumes/kubernetes.io~projected/kube-api-access-chhmv:{mountpoint:/var/lib/kubelet/pods/31ddd1aa-9d35-47bd-9553-9538f3517707/volumes/kubernetes.io~projected/kube-api-access-chhmv major:0 minor:345 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/31ddd1aa-9d35-47bd-9553-9538f3517707/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/31ddd1aa-9d35-47bd-9553-9538f3517707/volumes/kubernetes.io~secret/signing-key major:0 minor:342 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/350f9704-1e0e-4f41-958b-8b0b16761375/volumes/kubernetes.io~projected/kube-api-access-rrdbd:{mountpoint:/var/lib/kubelet/pods/350f9704-1e0e-4f41-958b-8b0b16761375/volumes/kubernetes.io~projected/kube-api-access-rrdbd major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd/volumes/kubernetes.io~projected/kube-api-access-877j7:{mountpoint:/var/lib/kubelet/pods/5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd/volumes/kubernetes.io~projected/kube-api-access-877j7 major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/63784b03-969b-4e2a-b799-e027ad1c0183/volumes/kubernetes.io~projected/kube-api-access-zwkv6:{mountpoint:/var/lib/kubelet/pods/63784b03-969b-4e2a-b799-e027ad1c0183/volumes/kubernetes.io~projected/kube-api-access-zwkv6 major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c/volumes/kubernetes.io~projected/kube-api-access-qsr2l:{mountpoint:/var/lib/kubelet/pods/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c/volumes/kubernetes.io~projected/kube-api-access-qsr2l major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6eccdb56-305f-4a78-b02b-e3f983030542/volumes/kubernetes.io~projected/kube-api-access-nbjw5:{mountpoint:/var/lib/kubelet/pods/6eccdb56-305f-4a78-b02b-e3f983030542/volumes/kubernetes.io~projected/kube-api-access-nbjw5 major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7153f183-6e65-4aed-b021-d0a8c3070fce/volumes/kubernetes.io~projected/kube-api-access-hc28v:{mountpoint:/var/lib/kubelet/pods/7153f183-6e65-4aed-b021-d0a8c3070fce/volumes/kubernetes.io~projected/kube-api-access-hc28v major:0 minor:334 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ffa4cb8-b390-49d2-839d-3cf43397eb6f/volumes/kubernetes.io~projected/kube-api-access-4r5qh:{mountpoint:/var/lib/kubelet/pods/7ffa4cb8-b390-49d2-839d-3cf43397eb6f/volumes/kubernetes.io~projected/kube-api-access-4r5qh major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/837e8971-01d7-44f5-81bb-bf206d191f2d/volumes/kubernetes.io~projected/kube-api-access-m4tx9:{mountpoint:/var/lib/kubelet/pods/837e8971-01d7-44f5-81bb-bf206d191f2d/volumes/kubernetes.io~projected/kube-api-access-m4tx9 major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/837e8971-01d7-44f5-81bb-bf206d191f2d/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/837e8971-01d7-44f5-81bb-bf206d191f2d/volumes/kubernetes.io~secret/serving-cert major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93/volumes/kubernetes.io~projected/kube-api-access-vqgl9:{mountpoint:/var/lib/kubelet/pods/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93/volumes/kubernetes.io~projected/kube-api-access-vqgl9 major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4/volumes/kubernetes.io~projected/kube-api-access-m7sk4:{mountpoint:/var/lib/kubelet/pods/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4/volumes/kubernetes.io~projected/kube-api-access-m7sk4 major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8f11a401-ced0-47dc-8814-c23849ba794b/volumes/kubernetes.io~projected/kube-api-access-czvnq:{mountpoint:/var/lib/kubelet/pods/8f11a401-ced0-47dc-8814-c23849ba794b/volumes/kubernetes.io~projected/kube-api-access-czvnq major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8f11a401-ced0-47dc-8814-c23849ba794b/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/8f11a401-ced0-47dc-8814-c23849ba794b/volumes/kubernetes.io~secret/serving-cert major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/947dc777-7dd0-4244-913a-5dd068e6e6a3/volumes/kubernetes.io~projected/kube-api-access-9dpmm:{mountpoint:/var/lib/kubelet/pods/947dc777-7dd0-4244-913a-5dd068e6e6a3/volumes/kubernetes.io~projected/kube-api-access-9dpmm major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/947dc777-7dd0-4244-913a-5dd068e6e6a3/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/947dc777-7dd0-4244-913a-5dd068e6e6a3/volumes/kubernetes.io~secret/serving-cert major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a1c203ed-13a2-429b-8a07-34751a25ca06/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/a1c203ed-13a2-429b-8a07-34751a25ca06/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a1c203ed-13a2-429b-8a07-34751a25ca06/volumes/kubernetes.io~projected/kube-api-access-xhszk:{mountpoint:/var/lib/kubelet/pods/a1c203ed-13a2-429b-8a07-34751a25ca06/volumes/kubernetes.io~projected/kube-api-access-xhszk major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a1c203ed-13a2-429b-8a07-34751a25ca06/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/a1c203ed-13a2-429b-8a07-34751a25ca06/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a543490c-5c05-4468-a0ad-25ebb475e78e/volumes/kubernetes.io~projected/kube-api-access-4748v:{mountpoint:/var/lib/kubelet/pods/a543490c-5c05-4468-a0ad-25ebb475e78e/volumes/kubernetes.io~projected/kube-api-access-4748v major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ad5dcaa4-c226-4639-92fe-ace864d049ac/volumes/kubernetes.io~projected/kube-api-access-g2f44:{mountpoint:/var/lib/kubelet/pods/ad5dcaa4-c226-4639-92fe-ace864d049ac/volumes/kubernetes.io~projected/kube-api-access-g2f44 major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ad5dcaa4-c226-4639-92fe-ace864d049ac/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/ad5dcaa4-c226-4639-92fe-ace864d049ac/volumes/kubernetes.io~secret/serving-cert major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ae184786-7bc6-403a-87fc-185baf65ffa5/volumes/kubernetes.io~projected/kube-api-access-vtg4p:{mountpoint:/var/lib/kubelet/pods/ae184786-7bc6-403a-87fc-185baf65ffa5/volumes/kubernetes.io~projected/kube-api-access-vtg4p major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b368f5ac-f18a-49d2-b9be-6cca37b7505b/volumes/kubernetes.io~projected/kube-api-access-t4mr2:{mountpoint:/var/lib/kubelet/pods/b368f5ac-f18a-49d2-b9be-6cca37b7505b/volumes/kubernetes.io~projected/kube-api-access-t4mr2 major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b3b53bf9-0305-43ad-99b1-26f35c841020/volumes/kubernetes.io~projected/kube-api-access-9rdlx:{mountpoint:/var/lib/kubelet/pods/b3b53bf9-0305-43ad-99b1-26f35c841020/volumes/kubernetes.io~projected/kube-api-access-9rdlx major:0 minor:319 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b6ad3759-e11c-476d-82ee-fffdf8fd6d47/volumes/kubernetes.io~projected/kube-api-access-xqrk2:{mountpoint:/var/lib/kubelet/pods/b6ad3759-e11c-476d-82ee-fffdf8fd6d47/volumes/kubernetes.io~projected/kube-api-access-xqrk2 major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b6ad3759-e11c-476d-82ee-fffdf8fd6d47/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/b6ad3759-e11c-476d-82ee-fffdf8fd6d47/volumes/kubernetes.io~secret/default-certificate major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b6ad3759-e11c-476d-82ee-fffdf8fd6d47/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/b6ad3759-e11c-476d-82ee-fffdf8fd6d47/volumes/kubernetes.io~secret/stats-auth major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bf94acd7-605c-4d9a-9021-5ce66a126edd/volumes/kubernetes.io~projected/kube-api-access-swjdw:{mountpoint:/var/lib/kubelet/pods/bf94acd7-605c-4d9a-9021-5ce66a126edd/volumes/kubernetes.io~projected/kube-api-access-swjdw major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c652cee1-24f4-460a-8e1a-669b0ae9c0b1/volumes/kubernetes.io~projected/kube-api-access-87lrl:{mountpoint:/var/lib/kubelet/pods/c652cee1-24f4-460a-8e1a-669b0ae9c0b1/volumes/kubernetes.io~projected/kube-api-access-87lrl major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f/volumes/kubernetes.io~projected/kube-api-access-jlhz5:{mountpoint:/var/lib/kubelet/pods/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f/volumes/kubernetes.io~projected/kube-api-access-jlhz5 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c964bbd5-fa82-459f-b464-d27fd1080654/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/c964bbd5-fa82-459f-b464-d27fd1080654/volumes/kubernetes.io~secret/agent-certs major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf41b924-24b7-4a89-9d11-af8b4246fcde/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/cf41b924-24b7-4a89-9d11-af8b4246fcde/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fa19a031-f171-495b-a72d-f4f79b88fceb/volumes/kubernetes.io~projected/kube-api-access-rftvj:{mountpoint:/var/lib/kubelet/pods/fa19a031-f171-495b-a72d-f4f79b88fceb/volumes/kubernetes.io~projected/kube-api-access-rftvj major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fa19a031-f171-495b-a72d-f4f79b88fceb/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/fa19a031-f171-495b-a72d-f4f79b88fceb/volumes/kubernetes.io~secret/cert major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fce92e88-114c-49c7-b658-193d508126e9/volumes/kubernetes.io~projected/kube-api-access-shjmc:{mountpoint:/var/lib/kubelet/pods/fce92e88-114c-49c7-b658-193d508126e9/volumes/kubernetes.io~projected/kube-api-access-shjmc major:0 minor:314 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/d79fd1cc8f5910513f37f73473d149f868670ea977e6b4ec06b370b90b7071ac/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/182f9632f5e4a78ced7f11a37aef18d1e25e3237196246914b9aed5428aaea90/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/9d7ac41b9fbf9a77b8a158b6b53d8cf2fd8e01540bbc63d34f145da228e87e95/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-106:{mountpoint:/var/lib/containers/storage/overlay/f09ef6e6cc3c15a1efa9dba5658c26e9a0d1a339e3a9543f26732a5daf8d188f/merged major:0 minor:106 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/6ac5c43a472b1811c0aa2032755599ceb0283172600e2a0c9feaf9e02921e339/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/3af35a5607a650b651215155628a26ea3d872d5aed3635b3ea819b8d0f4d3e61/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/14c3ee28be79e9c7c97f456f45aa22dc6caa89498898202c5be97aa6f7775d42/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/071371b92bfeb421240c6d810191040b3484cee1a886d831b3d9e38eb8a80c66/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/b1bfcce3aa05c6c0743a5f308e31f534a22a5c982b8e00e4b7bdb5e268c59f28/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/a8dee7cecdd865f91b23b92154cae317dc79d6c534baaa737cd920ac6dc34af0/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-139:{mountpoint:/var/lib/containers/storage/overlay/65d82fdc1dbbc004e92d424baa05521e6a643a5ea3093693fa110c1af86a6fd7/merged major:0 minor:139 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/ed0365b8eff116400fbcacf10ec170e12a23f1282fac4641ff65829af3834149/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/c34d0919f47c2ee7d1456883296f94c4121bbf4bc7a81b351870bf3e35221b7f/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/a9f1ee502c8e12ccc632664fae88c08ac1c4617b94d83065824533b374eee0a0/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-158:{mountpoint:/var/lib/containers/storage/overlay/e2ab37b8f401de8f629cbf5a692108bb45cf711cf0ce8d1ab55a2f76d4ec6eab/merged major:0 minor:158 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/13e18aa55b2d1037bd59c0a90fdd6fc5d631b68983904bf80c8a114f0c64a733/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/232764ea5c02bd6cd81acf7bf2b8137c4db51ffc735d3e1c46df3a6fd28c968a/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/10e32a79263cd26a5ea2fb4b668af70a73df06a2cfcb6330fe7be64d221b6756/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/e9fe41853663dec7a0b19017506b12aac51787110d6604c308c1329722cb114a/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/a761a82fa045764f8a30f9951bf6ef42cba5ad761b7adebc87869d3247e2c9dd/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/49226b6f65a9cffdb5184f418614d1db6bae72b30ca641a10a67e3f88f7209b4/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/e5f6af585d9ae2bcb902a5f954e014c3606535f0c6f128d773daa73655259cd7/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/6960e96b02fd8701dfd645251b32e99cadef50fdde8cf5cbcf8f8821188653e5/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/4818b4b4eba58806dfe3034c7096436f58143a453aa3467b11def7f1f1287d63/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/24195c031ed36a7093b1865fdc3bc5b64cb7025307f0326b2df3f07b053409f7/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/6b8eefc36fa0bb0e6e2913ceadfa2b0a8a33f948b958526928d239b70f7e8815/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/f53656e4865c96ca9956ec7a1821b67dc508635f3da41f17d448cdb5e681b658/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/9154e44643e1268c28575c870aa4400cc7ed0772cb57e82861e8ca56d9b6fc0b/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/15c1ea709f06e88f5a316f6327d0b88e08cfa8a06f4a9d33272ae0cb393fb57a/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/6f97eaa5e7b0ed462e9504706def2d5271733612c5841248e2bf5b9b34b8a692/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/e616fec54880f37c27d30144ba0209f09552418df75e00717883d27585af8bff/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/b63e7d5c3acf3c2d79d116e952b36d978788bddd0b646e77e6ad53a9138477d7/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/da755b5464deb98edbfea8722d43f1d8e6b36ffe95b7bf662ee33bc569b15199/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/78670d5a9ee61291341a77a20149616e8d0ba2b315ed8f4791fc5e00a3b9a194/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-274:{mountpoint:/var/lib/containers/storage/overlay/1e4b404a4c8881282c0a7735eed37b39f4840765644ea5a6c489fb4c3e8f0c0f/merged major:0 minor:274 fsType:overlay blockSize:0} overlay_0-276:{mountpoint:/var/lib/containers/storage/overlay/d00a76f42dd41dad2a8914cd12ddc193930a847e7537b1a4a022436810c19337/merged major:0 minor:276 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/fcaa76b147c8942c850a639beb0bf9f891d9a5c30a7d2a3948bf9e262736f152/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/b268477030d0a5b999432c3568c50bfb44efc30ed546f75fafec2c3fa4526a40/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-312:{mountpoint:/var/lib/containers/storage/overlay/3d686f8c54b2b998b74fce49c60d9e42a1d6ca5f1e1ab4ea4365b125c95eda1c/merged major:0 minor:312 fsType:overlay blockSize:0} overlay_0-320:{mountpoint:/var/lib/containers/storage/overlay/ffbe8ecff6808cd8ea5c62c5d9d1abce9bd989e6c37513aeff613459d9b5ad86/merged major:0 minor:320 fsType:overlay blockSize:0} overlay_0-324:{mountpoint:/var/lib/containers/storage/overlay/a37e493e8c36be3f61535c6df80c9ae5e622b0be088784bf46b5c8b46d45d6b5/merged major:0 minor:324 fsType:overlay blockSize:0} overlay_0-326:{mountpoint:/var/lib/containers/storage/overlay/44ac6fdb18dadb5dbddfc8af2daac9ceb2cf059452653aaa8b025bd259cfb415/merged major:0 minor:326 fsType:overlay blockSize:0} overlay_0-331:{mountpoint:/var/lib/containers/storage/overlay/3e879c7db7ff4a6d578e2f4eec72832df23ae97aecb1c81bd56dc310f4170c45/merged major:0 minor:331 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/cc3bd34230f1c015ba5ebf083d25fe069900780a818b77adfffade49f02d162e/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-350:{mountpoint:/var/lib/containers/storage/overlay/542b167bc37edf1d7a46565fcb9d28ba55e6bf262ffc856bcd30094931869821/merged major:0 minor:350 fsType:overlay blockSize:0} overlay_0-352:{mountpoint:/var/lib/containers/storage/overlay/47d07eac3d5bab45e9c4cf339b4d688507d4664840d04d7b54f99243b300e271/merged major:0 minor:352 fsType:overlay blockSize:0} overlay_0-359:{mountpoint:/var/lib/containers/storage/overlay/9414bba82a00e49d0bc03c2735bae7e2e3d348293282b20cbe53ec2a4a7e7c15/merged major:0 minor:359 fsType:overlay blockSize:0} overlay_0-361:{mountpoint:/var/lib/containers/storage/overlay/4077211d6b348e00a44229344e3eb83d3d07d1462165813225c9ecfe7e8dd75c/merged major:0 minor:361 fsType:overlay blockSize:0} overlay_0-363:{mountpoint:/var/lib/containers/storage/overlay/dbe4157b784b4b4d0cfef276b3a853032cf8f0f4aaea156beabc405235c28fbb/merged major:0 minor:363 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/3819632c118a2ef38600a4612101ff6bd6114ed74df52e3c362f8b9e2c286983/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/80ef0427d7a8181b5fdef67ab99bb2baba4867565cd940113335a63a4012ee0a/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-380:{mountpoint:/var/lib/containers/storage/overlay/fe21b767fbd9398f65fdbee6187b87d8d7baa50aae976f6cf94b1760f03119a3/merged major:0 minor:380 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/7cbe2d649f03e49546342368bde75f5656738a52bbb643d35f367e3b0e183799/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/4aad15ffd5a1c88014b6b0a88e33d0d08c2c2800188cb5cf1ec02a02ded35f98/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/30f7181184a5eeb1887162c56d0dd189f6765aa2388270f83bf3518454ee5e75/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/411981fda6e41d1b80492b17f844accf14c0f8dbd5b9632a034ce5f0be7cd169/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/86f75d3b0b6b26d3fbc47d429b532c827a2b6ae4a04b821e55e3a2b63e7884c7/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/e9521371d8bf9638b55f2d963ab063f94e1fb9d77afbf87f24e17e4eec8f5277/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/8fcd55f1205a30dfc3f2f903c5e674e6b6093dd9195e2ef0c9ebc3869ad901a2/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/77f8dc68373110f67952294028468df9917a63f4137ccd47b0f7fa2603955ce5/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/19564fc5c470cbfd8cdfd5b331f20f485a65264353c22f0bd495389d52d4698c/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/64a755090c062a0702333b075078b8c22be1b1e38d3af934cbafa7be52aaf2ee/merged major:0 minor:96 fsType:overlay blockSize:0}] Oct 29 23:12:52.711964 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.711583 5056 manager.go:217] Machine: {Timestamp:2026-10-29 23:12:52.711089717 +0000 UTC m=+0.077837340 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102182 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec265682fb0b989e8fef09d21f55ab99 SystemUUID:ec265682-fb0b-989e-8fef-09d21f55ab99 BootID:d1d15794-869e-496d-b1b6-185e9854df6a Filesystems:[{Device:/var/lib/kubelet/pods/a1c203ed-13a2-429b-8a07-34751a25ca06/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a8b4722e4e6b2bb95dc0ce5220d860476ec452f37aa982d55f570cc0d55a8c91/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b6ad3759-e11c-476d-82ee-fffdf8fd6d47/volumes/kubernetes.io~projected/kube-api-access-xqrk2 DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/82353c5785c533809f1de6dd9818f418c39cccf47e5cde5e41675d3cd4767a9f/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-326 DeviceMajor:0 DeviceMinor:326 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/91e59f285c148ed7601fd08e73b9fdd80c84a9a2e14036b88020ad258d7218ea/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/50b667f42d34af62fd160fc85feff19064c5a64317e6ce286a2a267e8bf72c07/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ad5dcaa4-c226-4639-92fe-ace864d049ac/volumes/kubernetes.io~projected/kube-api-access-g2f44 DeviceMajor:0 DeviceMinor:226 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e4cbf370e140b1541199154f795361d6b0d7b53f23bc73b3585577e1c9f6fd0e/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/13ed2c84b37ccdceedcaa24fb57881ee76e650cdf5ba3719e10e76572a6873d8/userdata/shm DeviceMajor:0 DeviceMinor:300 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-331 DeviceMajor:0 DeviceMinor:331 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/15fd926f7c91f01a1e15e40ea2b2d3c9949fad4f8bed6f1cec57a3c827730f16/userdata/shm DeviceMajor:0 DeviceMinor:228 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c652cee1-24f4-460a-8e1a-669b0ae9c0b1/volumes/kubernetes.io~projected/kube-api-access-87lrl DeviceMajor:0 DeviceMinor:66 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c/volumes/kubernetes.io~projected/kube-api-access-qsr2l DeviceMajor:0 DeviceMinor:216 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-312 DeviceMajor:0 DeviceMinor:312 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a1c203ed-13a2-429b-8a07-34751a25ca06/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-320 DeviceMajor:0 DeviceMinor:320 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/350f9704-1e0e-4f41-958b-8b0b16761375/volumes/kubernetes.io~projected/kube-api-access-rrdbd DeviceMajor:0 DeviceMinor:218 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:227 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ce41cdd0cdfb13694d7dcafdb1f3ce7ec1f1caf602a4d21511fdef6c0af294fd/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/63784b03-969b-4e2a-b799-e027ad1c0183/volumes/kubernetes.io~projected/kube-api-access-zwkv6 DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/bf94acd7-605c-4d9a-9021-5ce66a126edd/volumes/kubernetes.io~projected/kube-api-access-swjdw DeviceMajor:0 DeviceMinor:205 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a2a8836701e6ddf02d832bc5bd6c5d5964e776520c2c9cec1a5f49d7e98774cd/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-276 DeviceMajor:0 DeviceMinor:276 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ad5dcaa4-c226-4639-92fe-ace864d049ac/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:210 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4a780d6c876c96b2ac229f39a9eb1bc78c74918bd006b1fb2016e24048655c15/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93/volumes/kubernetes.io~projected/kube-api-access-vqgl9 DeviceMajor:0 DeviceMinor:221 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/31ddd1aa-9d35-47bd-9553-9538f3517707/volumes/kubernetes.io~projected/kube-api-access-chhmv DeviceMajor:0 DeviceMinor:345 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ae184786-7bc6-403a-87fc-185baf65ffa5/volumes/kubernetes.io~projected/kube-api-access-vtg4p DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:208 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fbe63081c629c2739579aaea9f2d9cd911ded08084074e23ea18ea78aac22f30/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/b3b53bf9-0305-43ad-99b1-26f35c841020/volumes/kubernetes.io~projected/kube-api-access-9rdlx DeviceMajor:0 DeviceMinor:319 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/382bd5f104a5df266b157f80d862e46f2a5dc5db186f38c34a273b90b216499e/userdata/shm DeviceMajor:0 DeviceMinor:340 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/31ddd1aa-9d35-47bd-9553-9538f3517707/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:342 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/25afb96c-6351-470d-8dfa-dd8a9145be40/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:217 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cf41b924-24b7-4a89-9d11-af8b4246fcde/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:294 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9c3adfcc48550f5cacb81cf8eb7f1bdd2f3ef037fb5cd65766debb6790819d43/userdata/shm DeviceMajor:0 DeviceMinor:322 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/21385963-6be4-4e3f-a996-f48b0c9ab0b1/volumes/kubernetes.io~projected/kube-api-access-jfhfl DeviceMajor:0 DeviceMinor:60 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-106 DeviceMajor:0 DeviceMinor:106 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/837e8971-01d7-44f5-81bb-bf206d191f2d/volumes/kubernetes.io~projected/kube-api-access-m4tx9 DeviceMajor:0 DeviceMinor:219 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd/volumes/kubernetes.io~projected/kube-api-access-877j7 DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/25afb96c-6351-470d-8dfa-dd8a9145be40/volumes/kubernetes.io~projected/kube-api-access-bzc9p DeviceMajor:0 DeviceMinor:222 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fce92e88-114c-49c7-b658-193d508126e9/volumes/kubernetes.io~projected/kube-api-access-shjmc DeviceMajor:0 DeviceMinor:314 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2656c9a13e8e7c08726889a49491f091ce0278df09e873d2e1ba0bb4b77c81fc/userdata/shm DeviceMajor:0 DeviceMinor:343 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/25afb96c-6351-470d-8dfa-dd8a9145be40/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:209 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e008b9fb8b4d2bb09c28afa47ec76dad17bd4525da1dc9650d2142c524970493/userdata/shm DeviceMajor:0 DeviceMinor:305 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-363 DeviceMajor:0 DeviceMinor:363 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e2e3cb8499a07a3b027a62e49940d2b429c696b285967e574f469eecea1d3bb2/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/837e8971-01d7-44f5-81bb-bf206d191f2d/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:213 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e2f99c191ef1a42bf8aee30eaaa041e3a03f7536a34173bd03dc701128e6b7a7/userdata/shm DeviceMajor:0 DeviceMinor:243 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-274 DeviceMajor:0 DeviceMinor:274 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b368f5ac-f18a-49d2-b9be-6cca37b7505b/volumes/kubernetes.io~projected/kube-api-access-t4mr2 DeviceMajor:0 DeviceMinor:295 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f4267b48ecd99c04495ab3c0a8971ee0931c33620e4b6c0e69d7338f0365f59a/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8f11a401-ced0-47dc-8814-c23849ba794b/volumes/kubernetes.io~projected/kube-api-access-czvnq DeviceMajor:0 DeviceMinor:223 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1575b8c7-d3a3-4956-bb56-72ba26dae4f6/volumes/kubernetes.io~projected/kube-api-access-fnjnb DeviceMajor:0 DeviceMinor:224 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-352 DeviceMajor:0 DeviceMinor:352 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-139 DeviceMajor:0 DeviceMinor:139 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-350 DeviceMajor:0 DeviceMinor:350 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7ffa4cb8-b390-49d2-839d-3cf43397eb6f/volumes/kubernetes.io~projected/kube-api-access-4r5qh DeviceMajor:0 DeviceMinor:62 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b6ad3759-e11c-476d-82ee-fffdf8fd6d47/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/141e20972e4b1ee800df8a6bdcb140a79bc979449f8cb50efcf252d7f2566176/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-361 DeviceMajor:0 DeviceMinor:361 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c964bbd5-fa82-459f-b464-d27fd1080654/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b6ad3759-e11c-476d-82ee-fffdf8fd6d47/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/947dc777-7dd0-4244-913a-5dd068e6e6a3/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:206 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8f11a401-ced0-47dc-8814-c23849ba794b/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:207 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/07425f46-dc9b-4d82-a1b8-246162054ea9/volumes/kubernetes.io~projected/kube-api-access-b24kn DeviceMajor:0 DeviceMinor:339 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f/volumes/kubernetes.io~projected/kube-api-access-jlhz5 DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1211641d-1d2c-4de0-b993-5f1e84edc09f/volumes/kubernetes.io~projected/kube-api-access-gh8rg DeviceMajor:0 DeviceMinor:61 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/fa19a031-f171-495b-a72d-f4f79b88fceb/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:211 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/baedcc53bf54f0b96ed8967ff1465e6d9552452fdfc48192e1b26666f1976295/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/579027c84fc7add6a266a6a9736f729b2d15b74fc0c1f2f9317ffdd6d25c5d9d/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/6eccdb56-305f-4a78-b02b-e3f983030542/volumes/kubernetes.io~projected/kube-api-access-nbjw5 DeviceMajor:0 DeviceMinor:67 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7153f183-6e65-4aed-b021-d0a8c3070fce/volumes/kubernetes.io~projected/kube-api-access-hc28v DeviceMajor:0 DeviceMinor:334 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ee7d783cd1ab14cd5213846bce28f544afd16f712d3405efe87a229a08e0e6c3/userdata/shm DeviceMajor:0 DeviceMinor:70 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/93eba272389667f4872d3f6c71e1d9639fc8144342dfee0a04b23f2d93508d3c/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fa19a031-f171-495b-a72d-f4f79b88fceb/volumes/kubernetes.io~projected/kube-api-access-rftvj DeviceMajor:0 DeviceMinor:220 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-324 DeviceMajor:0 DeviceMinor:324 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-380 DeviceMajor:0 DeviceMinor:380 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d94ef0ff71197b3c3c5133f0406808e8cc230c141502ffeca63e12e7de44b305/userdata/shm DeviceMajor:0 DeviceMinor:89 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:214 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/947dc777-7dd0-4244-913a-5dd068e6e6a3/volumes/kubernetes.io~projected/kube-api-access-9dpmm DeviceMajor:0 DeviceMinor:225 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ff4dfe98edc9d895bbc191f3efdfe4fd60c8092d4a11d6abf41b31fcaff7ebf5/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a/volumes/kubernetes.io~projected/kube-api-access-qf5x4 DeviceMajor:0 DeviceMinor:65 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c6475c759ad10fe4752007d759540cf86ea6626d354b9e60a62e848556883ae2/userdata/shm DeviceMajor:0 DeviceMinor:346 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/25afb96c-6351-470d-8dfa-dd8a9145be40/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:212 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4/volumes/kubernetes.io~projected/kube-api-access-m7sk4 DeviceMajor:0 DeviceMinor:215 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/a1c203ed-13a2-429b-8a07-34751a25ca06/volumes/kubernetes.io~projected/kube-api-access-xhszk DeviceMajor:0 DeviceMinor:64 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-359 DeviceMajor:0 DeviceMinor:359 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a543490c-5c05-4468-a0ad-25ebb475e78e/volumes/kubernetes.io~projected/kube-api-access-4748v DeviceMajor:0 DeviceMinor:63 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-158 DeviceMajor:0 DeviceMinor:158 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:13ed2c84b37ccdc MacAddress:de:23:95:4f:26:86 Speed:10000 Mtu:8901} {Name:141e20972e4b1ee MacAddress:d2:4d:40:1c:5a:bf Speed:10000 Mtu:8901} {Name:15fd926f7c91f01 MacAddress:2a:72:8f:41:a2:14 Speed:10000 Mtu:8901} {Name:2656c9a13e8e7c0 MacAddress:0a:f0:3f:da:21:09 Speed:10000 Mtu:8901} {Name:382bd5f104a5df2 MacAddress:3e:35:45:aa:94:21 Speed:10000 Mtu:8901} {Name:82353c5785c5338 MacAddress:62:3c:10:e7:d4:31 Speed:10000 Mtu:8901} {Name:9c3adfcc48550f5 MacAddress:52:12:ea:ec:59:a2 Speed:10000 Mtu:8901} {Name:a2a8836701e6ddf MacAddress:5e:00:d9:ca:f7:b7 Speed:10000 Mtu:8901} {Name:baedcc53bf54f0b MacAddress:1a:0d:d9:8f:b9:b1 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:c0:a6:21:ab Speed:0 Mtu:9001} {Name:br-int MacAddress:7a:d0:c0:ce:de:d9 Speed:0 Mtu:8901} {Name:c6475c759ad10fe MacAddress:3e:61:53:4b:31:16 Speed:10000 Mtu:8901} {Name:ce41cdd0cdfb136 MacAddress:ee:b6:be:06:9a:56 Speed:10000 Mtu:8901} {Name:e008b9fb8b4d2bb MacAddress:fe:8c:e6:3e:8f:c5 Speed:10000 Mtu:8901} {Name:e2f99c191ef1a42 MacAddress:32:61:be:05:ba:c0 Speed:10000 Mtu:8901} {Name:e4cbf370e140b15 MacAddress:6e:56:76:a2:00:27 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:c0:a6:21:ab Speed:0 Mtu:9001} {Name:fbe63081c629c27 MacAddress:82:26:76:61:73:f4 Speed:10000 Mtu:8901} {Name:ff4dfe98edc9d89 MacAddress:a2:8f:bb:c3:ec:7d Speed:10000 Mtu:8901} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:4e:72:db:26:c4:80 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 29 23:12:52.712175 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.711968 5056 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 29 23:12:52.712175 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.712015 5056 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 29 23:12:52.712386 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.712367 5056 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 29 23:12:52.712510 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.712387 5056 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-86.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 29 23:12:52.712554 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.712512 5056 topology_manager.go:143] "Creating topology manager with none policy" Oct 29 23:12:52.712554 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.712521 5056 container_manager_linux.go:311] "Creating device plugin manager" Oct 29 23:12:52.712554 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.712532 5056 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 29 23:12:52.712554 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.712545 5056 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 29 23:12:52.712659 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.712563 5056 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 29 23:12:52.712720 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.712710 5056 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 29 23:12:52.712835 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.712827 5056 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 29 23:12:52.712870 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.712863 5056 kubelet.go:493] "Attempting to sync node with API server" Oct 29 23:12:52.712899 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.712874 5056 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 29 23:12:52.712899 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.712886 5056 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 29 23:12:52.712899 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.712895 5056 kubelet.go:405] "Adding apiserver pod source" Oct 29 23:12:52.712988 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.712904 5056 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 29 23:12:52.713714 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.713698 5056 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 29 23:12:52.713714 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.713710 5056 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 29 23:12:52.714253 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.714229 5056 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 29 23:12:52.716982 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.716963 5056 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 29 23:12:52.716982 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.716978 5056 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 29 23:12:52.717146 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.717099 5056 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Oct 29 23:12:52.717364 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.717348 5056 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 29 23:12:52.717437 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.717388 5056 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 29 23:12:52.717532 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.717520 5056 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 29 23:12:52.717585 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.717548 5056 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 29 23:12:52.717585 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.717559 5056 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 29 23:12:52.717585 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.717569 5056 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 29 23:12:52.717585 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.717578 5056 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 29 23:12:52.717772 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.717587 5056 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 29 23:12:52.717772 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.717597 5056 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 29 23:12:52.717772 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.717606 5056 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 29 23:12:52.717772 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.717623 5056 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 29 23:12:52.717772 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.717633 5056 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 29 23:12:52.717772 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.717647 5056 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 29 23:12:52.717772 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.717661 5056 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 29 23:12:52.717772 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.717700 5056 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 29 23:12:52.717772 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.717710 5056 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 29 23:12:52.721305 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.721290 5056 server.go:1263] "Started kubelet" Oct 29 23:12:52.721411 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.721355 5056 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 29 23:12:52.722595 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.722382 5056 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 29 23:12:52.722739 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.722724 5056 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 29 23:12:52.723153 ip-10-0-130-86 systemd[1]: Started Kubernetes Kubelet. Oct 29 23:12:52.724356 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.723608 5056 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 29 23:12:52.726317 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.726301 5056 server.go:317] "Adding debug handlers to kubelet server" Oct 29 23:12:52.728610 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.728590 5056 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 29 23:12:52.728689 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.728617 5056 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 29 23:12:52.728689 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.728672 5056 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-28 23:07:13 +0000 UTC" deadline="2028-06-09 08:57:34.722142542 +0000 UTC" Oct 29 23:12:52.728795 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.728690 5056 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14121h44m41.993454866s" Oct 29 23:12:52.728795 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.728746 5056 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 29 23:12:52.728894 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.728804 5056 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 29 23:12:52.729610 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.729591 5056 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 29 23:12:52.729685 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:52.729636 5056 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-130-86.ec2.internal\" not found" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.730603 5056 factory.go:55] Registering systemd factory Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.730631 5056 factory.go:223] Registration of the systemd container factory successfully Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.731549 5056 factory.go:153] Registering CRI-O factory Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.731566 5056 factory.go:223] Registration of the crio container factory successfully Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.731620 5056 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.731644 5056 factory.go:103] Registering Raw factory Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.731664 5056 manager.go:1199] Started watching for new ooms in manager Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732029 5056 manager.go:319] Starting recovery of all containers Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732135 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="31ddd1aa-9d35-47bd-9553-9538f3517707" volumeName="kubernetes.io/projected/31ddd1aa-9d35-47bd-9553-9538f3517707-kube-api-access-chhmv" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732201 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="31ddd1aa-9d35-47bd-9553-9538f3517707" volumeName="kubernetes.io/secret/31ddd1aa-9d35-47bd-9553-9538f3517707-signing-key" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732217 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad5dcaa4-c226-4639-92fe-ace864d049ac" volumeName="kubernetes.io/secret/ad5dcaa4-c226-4639-92fe-ace864d049ac-serving-cert" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732230 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae184786-7bc6-403a-87fc-185baf65ffa5" volumeName="kubernetes.io/projected/ae184786-7bc6-403a-87fc-185baf65ffa5-kube-api-access-vtg4p" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732243 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b368f5ac-f18a-49d2-b9be-6cca37b7505b" volumeName="kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732253 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6ad3759-e11c-476d-82ee-fffdf8fd6d47" volumeName="kubernetes.io/projected/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-kube-api-access-xqrk2" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732315 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c652cee1-24f4-460a-8e1a-669b0ae9c0b1" volumeName="kubernetes.io/projected/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-kube-api-access-87lrl" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732329 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c964bbd5-fa82-459f-b464-d27fd1080654" volumeName="kubernetes.io/configmap/c964bbd5-fa82-459f-b464-d27fd1080654-konnectivity-ca" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732344 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a" volumeName="kubernetes.io/empty-dir/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-tuned" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732356 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21385963-6be4-4e3f-a996-f48b0c9ab0b1" volumeName="kubernetes.io/projected/21385963-6be4-4e3f-a996-f48b0c9ab0b1-kube-api-access-jfhfl" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732369 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="25afb96c-6351-470d-8dfa-dd8a9145be40" volumeName="kubernetes.io/empty-dir/25afb96c-6351-470d-8dfa-dd8a9145be40-ca-trust-extracted" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732380 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6eccdb56-305f-4a78-b02b-e3f983030542" volumeName="kubernetes.io/projected/6eccdb56-305f-4a78-b02b-e3f983030542-kube-api-access-nbjw5" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732392 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f11a401-ced0-47dc-8814-c23849ba794b" volumeName="kubernetes.io/projected/8f11a401-ced0-47dc-8814-c23849ba794b-kube-api-access-czvnq" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732405 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad5dcaa4-c226-4639-92fe-ace864d049ac" volumeName="kubernetes.io/projected/ad5dcaa4-c226-4639-92fe-ace864d049ac-kube-api-access-g2f44" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732419 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fce92e88-114c-49c7-b658-193d508126e9" volumeName="kubernetes.io/empty-dir/fce92e88-114c-49c7-b658-193d508126e9-archives-path" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732430 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="25afb96c-6351-470d-8dfa-dd8a9145be40" volumeName="kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-certificates" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732442 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="947dc777-7dd0-4244-913a-5dd068e6e6a3" volumeName="kubernetes.io/projected/947dc777-7dd0-4244-913a-5dd068e6e6a3-kube-api-access-9dpmm" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732481 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b218e5d9-83c0-4976-90da-471e6b86d460" volumeName="kubernetes.io/configmap/b218e5d9-83c0-4976-90da-471e6b86d460-nginx-conf" seLinuxMountContext="" Oct 29 23:12:52.732483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732497 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c964bbd5-fa82-459f-b464-d27fd1080654" volumeName="kubernetes.io/secret/c964bbd5-fa82-459f-b464-d27fd1080654-agent-certs" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732514 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a" volumeName="kubernetes.io/empty-dir/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-tmp" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732527 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1575b8c7-d3a3-4956-bb56-72ba26dae4f6" volumeName="kubernetes.io/projected/1575b8c7-d3a3-4956-bb56-72ba26dae4f6-kube-api-access-fnjnb" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732540 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21385963-6be4-4e3f-a996-f48b0c9ab0b1" volumeName="kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-whereabouts-flatfile-configmap" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732556 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" volumeName="kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-trusted-ca" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732568 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" volumeName="kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-bound-sa-token" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732580 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6eccdb56-305f-4a78-b02b-e3f983030542" volumeName="kubernetes.io/configmap/6eccdb56-305f-4a78-b02b-e3f983030542-iptables-alerter-script" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732592 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad5dcaa4-c226-4639-92fe-ace864d049ac" volumeName="kubernetes.io/configmap/ad5dcaa4-c226-4639-92fe-ace864d049ac-service-ca-bundle" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732605 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad5dcaa4-c226-4639-92fe-ace864d049ac" volumeName="kubernetes.io/empty-dir/ad5dcaa4-c226-4639-92fe-ace864d049ac-snapshots" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732616 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" volumeName="kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-certificates" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732634 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f11a401-ced0-47dc-8814-c23849ba794b" volumeName="kubernetes.io/secret/8f11a401-ced0-47dc-8814-c23849ba794b-serving-cert" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732646 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="947dc777-7dd0-4244-913a-5dd068e6e6a3" volumeName="kubernetes.io/configmap/947dc777-7dd0-4244-913a-5dd068e6e6a3-trusted-ca" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732660 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="947dc777-7dd0-4244-913a-5dd068e6e6a3" volumeName="kubernetes.io/configmap/947dc777-7dd0-4244-913a-5dd068e6e6a3-config" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732671 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="947dc777-7dd0-4244-913a-5dd068e6e6a3" volumeName="kubernetes.io/secret/947dc777-7dd0-4244-913a-5dd068e6e6a3-serving-cert" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732687 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c6fc8782-2d71-4cb3-ab4b-b1d14aad460f" volumeName="kubernetes.io/projected/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-kube-api-access-jlhz5" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732701 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fce92e88-114c-49c7-b658-193d508126e9" volumeName="kubernetes.io/projected/fce92e88-114c-49c7-b658-193d508126e9-kube-api-access-shjmc" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732714 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1c203ed-13a2-429b-8a07-34751a25ca06" volumeName="kubernetes.io/projected/a1c203ed-13a2-429b-8a07-34751a25ca06-kube-api-access-xhszk" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732725 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad5dcaa4-c226-4639-92fe-ace864d049ac" volumeName="kubernetes.io/empty-dir/ad5dcaa4-c226-4639-92fe-ace864d049ac-tmp" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732736 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="25afb96c-6351-470d-8dfa-dd8a9145be40" volumeName="kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-bound-sa-token" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732750 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1c203ed-13a2-429b-8a07-34751a25ca06" volumeName="kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-ovnkube-config" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732787 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ad5dcaa4-c226-4639-92fe-ace864d049ac" volumeName="kubernetes.io/configmap/ad5dcaa4-c226-4639-92fe-ace864d049ac-trusted-ca-bundle" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732803 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3b53bf9-0305-43ad-99b1-26f35c841020" volumeName="kubernetes.io/projected/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-api-access-9rdlx" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732815 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63784b03-969b-4e2a-b799-e027ad1c0183" volumeName="kubernetes.io/projected/63784b03-969b-4e2a-b799-e027ad1c0183-kube-api-access-zwkv6" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732827 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7153f183-6e65-4aed-b021-d0a8c3070fce" volumeName="kubernetes.io/projected/7153f183-6e65-4aed-b021-d0a8c3070fce-kube-api-access-hc28v" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732839 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8388282a-8b2c-4f5c-ab3d-ec58ea20cc93" volumeName="kubernetes.io/projected/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-kube-api-access-vqgl9" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732851 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf94acd7-605c-4d9a-9021-5ce66a126edd" volumeName="kubernetes.io/projected/bf94acd7-605c-4d9a-9021-5ce66a126edd-kube-api-access-swjdw" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732863 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf41b924-24b7-4a89-9d11-af8b4246fcde" volumeName="kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732876 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fa19a031-f171-495b-a72d-f4f79b88fceb" volumeName="kubernetes.io/secret/fa19a031-f171-495b-a72d-f4f79b88fceb-cert" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732889 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="25afb96c-6351-470d-8dfa-dd8a9145be40" volumeName="kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-kube-api-access-bzc9p" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732901 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fa19a031-f171-495b-a72d-f4f79b88fceb" volumeName="kubernetes.io/projected/fa19a031-f171-495b-a72d-f4f79b88fceb-kube-api-access-rftvj" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732913 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a" volumeName="kubernetes.io/projected/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-kube-api-access-qf5x4" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732925 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="25afb96c-6351-470d-8dfa-dd8a9145be40" volumeName="kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-installation-pull-secrets" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732937 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" volumeName="kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-installation-pull-secrets" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732948 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8388282a-8b2c-4f5c-ab3d-ec58ea20cc93" volumeName="kubernetes.io/empty-dir/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-tmp-dir" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732962 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a543490c-5c05-4468-a0ad-25ebb475e78e" volumeName="kubernetes.io/projected/a543490c-5c05-4468-a0ad-25ebb475e78e-kube-api-access-4748v" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732974 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fce92e88-114c-49c7-b658-193d508126e9" volumeName="kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-service-ca-bundle" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732987 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="07425f46-dc9b-4d82-a1b8-246162054ea9" volumeName="kubernetes.io/projected/07425f46-dc9b-4d82-a1b8-246162054ea9-kube-api-access-b24kn" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.732999 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1211641d-1d2c-4de0-b993-5f1e84edc09f" volumeName="kubernetes.io/projected/1211641d-1d2c-4de0-b993-5f1e84edc09f-kube-api-access-gh8rg" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733020 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21385963-6be4-4e3f-a996-f48b0c9ab0b1" volumeName="kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cni-sysctl-allowlist" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733032 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="350f9704-1e0e-4f41-958b-8b0b16761375" volumeName="kubernetes.io/projected/350f9704-1e0e-4f41-958b-8b0b16761375-kube-api-access-rrdbd" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733045 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1c203ed-13a2-429b-8a07-34751a25ca06" volumeName="kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-ovnkube-script-lib" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733057 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a543490c-5c05-4468-a0ad-25ebb475e78e" volumeName="kubernetes.io/configmap/a543490c-5c05-4468-a0ad-25ebb475e78e-cni-binary-copy" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733069 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c652cee1-24f4-460a-8e1a-669b0ae9c0b1" volumeName="kubernetes.io/configmap/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-serviceca" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733082 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="25afb96c-6351-470d-8dfa-dd8a9145be40" volumeName="kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-image-registry-private-configuration" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733093 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f11a401-ced0-47dc-8814-c23849ba794b" volumeName="kubernetes.io/configmap/8f11a401-ced0-47dc-8814-c23849ba794b-config" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733107 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1c203ed-13a2-429b-8a07-34751a25ca06" volumeName="kubernetes.io/secret/a1c203ed-13a2-429b-8a07-34751a25ca06-ovn-node-metrics-cert" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733120 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fce92e88-114c-49c7-b658-193d508126e9" volumeName="kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-trusted-ca-bundle" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733131 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="21385963-6be4-4e3f-a996-f48b0c9ab0b1" volumeName="kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cni-binary-copy" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733142 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="837e8971-01d7-44f5-81bb-bf206d191f2d" volumeName="kubernetes.io/secret/837e8971-01d7-44f5-81bb-bf206d191f2d-serving-cert" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733153 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b3b53bf9-0305-43ad-99b1-26f35c841020" volumeName="kubernetes.io/empty-dir/b3b53bf9-0305-43ad-99b1-26f35c841020-data-volume" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733165 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6ad3759-e11c-476d-82ee-fffdf8fd6d47" volumeName="kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-default-certificate" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733176 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd" volumeName="kubernetes.io/projected/5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd-kube-api-access-877j7" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733190 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63784b03-969b-4e2a-b799-e027ad1c0183" volumeName="kubernetes.io/configmap/63784b03-969b-4e2a-b799-e027ad1c0183-telemetry-config" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733201 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" volumeName="kubernetes.io/empty-dir/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-ca-trust-extracted" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733213 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" volumeName="kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-kube-api-access-qsr2l" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733224 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4" volumeName="kubernetes.io/projected/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-kube-api-access-m7sk4" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733236 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b6ad3759-e11c-476d-82ee-fffdf8fd6d47" volumeName="kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-stats-auth" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733248 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="25afb96c-6351-470d-8dfa-dd8a9145be40" volumeName="kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-trusted-ca" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733278 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" volumeName="kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-image-registry-private-configuration" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733292 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8388282a-8b2c-4f5c-ab3d-ec58ea20cc93" volumeName="kubernetes.io/configmap/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-config-volume" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733304 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1211641d-1d2c-4de0-b993-5f1e84edc09f" volumeName="kubernetes.io/empty-dir/1211641d-1d2c-4de0-b993-5f1e84edc09f-tmp-dir" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733316 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="31ddd1aa-9d35-47bd-9553-9538f3517707" volumeName="kubernetes.io/configmap/31ddd1aa-9d35-47bd-9553-9538f3517707-signing-cabundle" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733327 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ffa4cb8-b390-49d2-839d-3cf43397eb6f" volumeName="kubernetes.io/projected/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-kube-api-access-4r5qh" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733338 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="837e8971-01d7-44f5-81bb-bf206d191f2d" volumeName="kubernetes.io/configmap/837e8971-01d7-44f5-81bb-bf206d191f2d-config" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733355 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="837e8971-01d7-44f5-81bb-bf206d191f2d" volumeName="kubernetes.io/projected/837e8971-01d7-44f5-81bb-bf206d191f2d-kube-api-access-m4tx9" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733366 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4" volumeName="kubernetes.io/configmap/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-config" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733378 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a1c203ed-13a2-429b-8a07-34751a25ca06" volumeName="kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-env-overrides" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733402 5056 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a543490c-5c05-4468-a0ad-25ebb475e78e" volumeName="kubernetes.io/configmap/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-daemon-config" seLinuxMountContext="" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733413 5056 reconstruct.go:97] "Volume reconstruction finished" Oct 29 23:12:52.733763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.733421 5056 reconciler.go:29] "Reconciler: start to sync state" Oct 29 23:12:52.750049 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.750023 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/1.log" Oct 29 23:12:52.752592 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.752572 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/1.log" Oct 29 23:12:52.753556 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.753528 5056 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 29 23:12:52.762418 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.762394 5056 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 29 23:12:52.762418 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.762418 5056 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 29 23:12:52.762586 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.762441 5056 kubelet.go:2521] "Starting kubelet main sync loop" Oct 29 23:12:52.762586 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:52.762486 5056 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Oct 29 23:12:52.774041 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.774022 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/1.log" Oct 29 23:12:52.774156 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.774051 5056 generic.go:356] "Generic (PLEG): container finished" podID="947dc777-7dd0-4244-913a-5dd068e6e6a3" containerID="a32678eaae6ae6d7b889b7350cba11dc4d854c7dae6d8bfaa12ca78621468dbb" exitCode=255 Oct 29 23:12:52.785350 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.785327 5056 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 23:12:52.785486 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.785470 5056 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Oct 29 23:12:52.786469 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.786441 5056 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 23:12:52.793377 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.793346 5056 generic.go:356] "Generic (PLEG): container finished" podID="21385963-6be4-4e3f-a996-f48b0c9ab0b1" containerID="f494c37fbb7a64c0269ebee03482b5b95d888f0c60866d8cbfaff9fd48b872e2" exitCode=0 Oct 29 23:12:52.793377 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.793376 5056 generic.go:356] "Generic (PLEG): container finished" podID="21385963-6be4-4e3f-a996-f48b0c9ab0b1" containerID="b143e3f545d53f57349cdf10ea2361787b9b8e7bb85cca1764a20730f93d5219" exitCode=0 Oct 29 23:12:52.793495 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.793390 5056 generic.go:356] "Generic (PLEG): container finished" podID="21385963-6be4-4e3f-a996-f48b0c9ab0b1" containerID="1f9da28be6977a4561a105d6bd2ba8b40fa61f5d9d0517443fb01373f8adb128" exitCode=0 Oct 29 23:12:52.793495 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.793396 5056 generic.go:356] "Generic (PLEG): container finished" podID="21385963-6be4-4e3f-a996-f48b0c9ab0b1" containerID="6380f9bdf6abcbb9d9ce03a1588d92b96a0d504bbf26b72af2eddeae75d854a0" exitCode=0 Oct 29 23:12:52.793495 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.793402 5056 generic.go:356] "Generic (PLEG): container finished" podID="21385963-6be4-4e3f-a996-f48b0c9ab0b1" containerID="72d5bae21d7690a330bd75bfa6a185e5373724ca00c190e06d185b07fd3e8334" exitCode=0 Oct 29 23:12:52.793495 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.793407 5056 generic.go:356] "Generic (PLEG): container finished" podID="21385963-6be4-4e3f-a996-f48b0c9ab0b1" containerID="b4f82737db306c147ad7dfa3c215526641372f4f8708bb52a43238dfce60f50a" exitCode=0 Oct 29 23:12:52.800652 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.800593 5056 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 23:12:52.800821 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.800805 5056 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 29 23:12:52.818546 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.818481 5056 generic.go:356] "Generic (PLEG): container finished" podID="966fb6b19f49ae6dfc8b44441b931f34" containerID="9db485ec3f27847f847194b0e2f47c9c84d0de41a34532233cc52f1e059b4ef5" exitCode=0 Oct 29 23:12:52.832638 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.832617 5056 manager.go:324] Recovery completed Oct 29 23:12:52.854144 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.854115 5056 cpu_manager.go:226] "Starting" policy="none" Oct 29 23:12:52.854144 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.854141 5056 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 29 23:12:52.854370 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.854162 5056 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 29 23:12:52.854370 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.854355 5056 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Oct 29 23:12:52.854486 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.854371 5056 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Oct 29 23:12:52.854486 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.854402 5056 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Oct 29 23:12:52.854486 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.854410 5056 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Oct 29 23:12:52.854486 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.854418 5056 policy_none.go:50] "Start" Oct 29 23:12:52.854486 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.854432 5056 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 29 23:12:52.854486 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.854449 5056 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 29 23:12:52.854768 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.854688 5056 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Oct 29 23:12:52.854768 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.854700 5056 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Oct 29 23:12:52.854768 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.854708 5056 policy_none.go:44] "Start" Oct 29 23:12:52.859845 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.859816 5056 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 29 23:12:52.862791 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:52.862773 5056 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Oct 29 23:12:52.863501 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.863489 5056 manager.go:341] "Starting Device Plugin manager" Oct 29 23:12:52.863978 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:52.863517 5056 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 29 23:12:52.864059 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.863982 5056 server.go:87] "Starting device plugin registration server" Oct 29 23:12:52.864289 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.864276 5056 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 29 23:12:52.864361 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.864290 5056 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 29 23:12:52.864422 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.864387 5056 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 29 23:12:52.864569 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.864529 5056 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 29 23:12:52.864681 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.864574 5056 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 29 23:12:52.864681 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.864583 5056 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 29 23:12:52.864784 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.864637 5056 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-29T23:12:52.864563337Z","UUID":"794c1407-543a-497c-b460-847777ca9b3e","Handler":null,"Name":"","Endpoint":""} Oct 29 23:12:52.865830 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.865815 5056 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 29 23:12:52.865830 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:52.865825 5056 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 29 23:12:52.865978 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.865852 5056 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 29 23:12:52.964418 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.964381 5056 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 29 23:12:52.968175 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.968156 5056 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientMemory" Oct 29 23:12:52.968286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.968191 5056 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasNoDiskPressure" Oct 29 23:12:52.968286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.968205 5056 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-130-86.ec2.internal" event="NodeHasSufficientPID" Oct 29 23:12:52.968286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:52.968253 5056 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.007967 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.007933 5056 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.008087 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.008017 5056 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.063093 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.063038 5056 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal"] Oct 29 23:12:53.063763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.063731 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" event={"ID":"966fb6b19f49ae6dfc8b44441b931f34","Type":"ContainerStarted","Data":"8dc18baec274648967ab86ab667acdcaafc3bda7b6b480bf2e49fe9fa5623414"} Oct 29 23:12:53.063840 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.063769 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" event={"ID":"966fb6b19f49ae6dfc8b44441b931f34","Type":"ContainerDied","Data":"9db485ec3f27847f847194b0e2f47c9c84d0de41a34532233cc52f1e059b4ef5"} Oct 29 23:12:53.063840 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.063793 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" event={"ID":"966fb6b19f49ae6dfc8b44441b931f34","Type":"ContainerStarted","Data":"f4267b48ecd99c04495ab3c0a8971ee0931c33620e4b6c0e69d7338f0365f59a"} Oct 29 23:12:53.063840 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.063820 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" event={"ID":"47e5a52938e8a36be364af182a627291","Type":"ContainerStarted","Data":"a9861ee4714c733778bb4d729d3ef68bcf0ec1234ac4b2437cd43d75100546e6"} Oct 29 23:12:53.063840 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.063828 5056 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.064022 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.063890 5056 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.064022 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.063832 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" event={"ID":"47e5a52938e8a36be364af182a627291","Type":"ContainerStarted","Data":"579027c84fc7add6a266a6a9736f729b2d15b74fc0c1f2f9317ffdd6d25c5d9d"} Oct 29 23:12:53.089735 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.089663 5056 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 29 23:12:53.089735 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:53.089709 5056 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-86.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.090302 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.090291 5056 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 29 23:12:53.090367 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:53.090316 5056 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.186698 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.186663 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/966fb6b19f49ae6dfc8b44441b931f34-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal\" (UID: \"966fb6b19f49ae6dfc8b44441b931f34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.186698 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.186695 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/966fb6b19f49ae6dfc8b44441b931f34-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal\" (UID: \"966fb6b19f49ae6dfc8b44441b931f34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.186869 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.186717 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/47e5a52938e8a36be364af182a627291-config\") pod \"kube-apiserver-proxy-ip-10-0-130-86.ec2.internal\" (UID: \"47e5a52938e8a36be364af182a627291\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.287365 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.287323 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/966fb6b19f49ae6dfc8b44441b931f34-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal\" (UID: \"966fb6b19f49ae6dfc8b44441b931f34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.287365 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.287332 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/966fb6b19f49ae6dfc8b44441b931f34-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal\" (UID: \"966fb6b19f49ae6dfc8b44441b931f34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.287569 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.287414 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/966fb6b19f49ae6dfc8b44441b931f34-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal\" (UID: \"966fb6b19f49ae6dfc8b44441b931f34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.287569 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.287442 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/47e5a52938e8a36be364af182a627291-config\") pod \"kube-apiserver-proxy-ip-10-0-130-86.ec2.internal\" (UID: \"47e5a52938e8a36be364af182a627291\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.287569 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.287494 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/47e5a52938e8a36be364af182a627291-config\") pod \"kube-apiserver-proxy-ip-10-0-130-86.ec2.internal\" (UID: \"47e5a52938e8a36be364af182a627291\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.287569 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.287509 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/966fb6b19f49ae6dfc8b44441b931f34-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal\" (UID: \"966fb6b19f49ae6dfc8b44441b931f34\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.713252 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.713217 5056 apiserver.go:52] "Watching apiserver" Oct 29 23:12:53.757245 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.757216 5056 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 29 23:12:53.757881 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.757859 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-blg76","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pn894","kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal","openshift-dns/node-resolver-zr8dl","kube-system/kas-connection-checker-685895c7c9-z6qn8","kube-system/kas-connection-checker-685895c7c9-mjz4s","openshift-image-registry/image-registry-7cbdc4b876-9xjlm","openshift-multus/network-metrics-daemon-2xktq","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw","openshift-console-operator/console-operator-7cd56b8dbc-2gzh7","openshift-multus/multus-49s6q","openshift-kube-storage-version-migrator/migrator-66df5d496b-75lxf","openshift-network-console/networking-console-plugin-86c478bb88-b9jng","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c","openshift-insights/periodic-gathering-hhksv-vlwlb","openshift-kube-storage-version-migrator/migrator-66df5d496b-j6lj7","openshift-service-ca/service-ca-5fffc97bdf-t9kdp","openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal","openshift-multus/multus-additional-cni-plugins-xzb6b","openshift-image-registry/node-ca-jg249","openshift-dns/dns-default-gnwhl","kube-system/kas-connection-checker-685895c7c9-74vqh","kube-system/global-pull-secret-syncer-gz4mg","openshift-network-diagnostics/network-check-source-6f78d59f99-nhgxp","openshift-insights/insights-runtime-extractor-x975f","openshift-network-operator/iptables-alerter-c9pz5","openshift-ingress-canary/ingress-canary-t8vj4","openshift-image-registry/image-registry-5679dbd569-548pj","openshift-network-diagnostics/network-check-target-dn96d","kube-system/konnectivity-agent-25m4p","openshift-ingress/router-default-866787b6f9-24krt","openshift-insights/insights-operator-7c4986d449-m72dz","openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd","openshift-cluster-node-tuning-operator/tuned-wd9cn"] Oct 29 23:12:53.759005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.758987 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:53.759358 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.759334 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:53.760492 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.760474 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:53.760938 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.760912 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:53.762953 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.762935 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" Oct 29 23:12:53.765286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.765247 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:53.773613 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.773595 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.773937 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.773920 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.774730 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.774714 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:53.774893 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.774873 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Oct 29 23:12:53.775018 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.774943 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.775119 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.775091 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.775251 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.775230 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.775347 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.775300 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Oct 29 23:12:53.775430 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.775414 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Oct 29 23:12:53.775831 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.775814 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Oct 29 23:12:53.775899 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.775841 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Oct 29 23:12:53.776034 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.776012 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.776101 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.776053 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.776101 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.776088 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.776250 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.776235 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:53.776406 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.776252 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Oct 29 23:12:53.777045 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.777026 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.777141 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.777127 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Oct 29 23:12:53.777204 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.777187 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:53.777500 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.777485 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Oct 29 23:12:53.777582 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.777539 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Oct 29 23:12:53.777582 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.777567 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 29 23:12:53.777582 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.777577 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.777807 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.777792 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.777888 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.777868 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Oct 29 23:12:53.778051 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.778036 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 29 23:12:53.806165 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.806139 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Oct 29 23:12:53.806287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.806187 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.806287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.806192 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.806411 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.806283 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.806411 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.806374 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.806526 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.806454 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 29 23:12:53.806526 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.806499 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Oct 29 23:12:53.806526 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.806501 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Oct 29 23:12:53.806657 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.806606 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 29 23:12:53.806657 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.806606 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 29 23:12:53.806657 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.806620 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 29 23:12:53.806657 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.806606 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.806657 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.806635 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.806819 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.806718 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.807003 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.806988 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 29 23:12:53.807109 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.807097 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.807165 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.807108 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.807165 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.807122 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.807272 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.807163 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Oct 29 23:12:53.807272 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.807171 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Oct 29 23:12:53.807740 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.807726 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 29 23:12:53.807923 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.807907 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Oct 29 23:12:53.807980 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.807920 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 29 23:12:53.807980 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.807930 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.807980 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.807951 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 29 23:12:53.808110 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.808097 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.808303 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.808289 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.808430 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.808415 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.808484 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.808415 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 29 23:12:53.808484 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.808444 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7fjkb\"" Oct 29 23:12:53.808552 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.808488 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.808836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.808822 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kcr8p\"" Oct 29 23:12:53.808878 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.808861 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 29 23:12:53.808925 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.808910 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.808962 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.808945 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-h2str\"" Oct 29 23:12:53.808992 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.808968 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-m6t5c\"" Oct 29 23:12:53.809032 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.809001 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Oct 29 23:12:53.809388 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.809368 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.809443 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.809428 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.809487 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.809429 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.809487 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.809458 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.809572 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.809532 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.809572 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.809540 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 29 23:12:53.809668 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.809567 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Oct 29 23:12:53.809668 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.809629 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 29 23:12:53.809668 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.809650 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 29 23:12:53.809802 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.809679 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 29 23:12:53.809802 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.809697 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-hj644\"" Oct 29 23:12:53.809802 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.809744 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-zfk9z\"" Oct 29 23:12:53.809802 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.809753 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-96xtz\"" Oct 29 23:12:53.809802 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.809632 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cxqm7\"" Oct 29 23:12:53.809802 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.809773 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wscf4\"" Oct 29 23:12:53.810064 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.809981 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-88spp\"" Oct 29 23:12:53.810113 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.810081 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4lr6v\"" Oct 29 23:12:53.810158 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.810117 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.810158 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.810149 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.810351 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.810331 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vsvpd\"" Oct 29 23:12:53.810415 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.810349 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 29 23:12:53.810415 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.810386 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-lbwxf\"" Oct 29 23:12:53.810527 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.810414 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 29 23:12:53.810527 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.810434 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.810527 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.810518 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.810669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.810525 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.810669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.810535 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-rlkhk\"" Oct 29 23:12:53.810669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.810594 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 29 23:12:53.810669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.810622 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Oct 29 23:12:53.810837 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.810687 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-5wvkd\"" Oct 29 23:12:53.810886 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.810856 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.810938 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.810921 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Oct 29 23:12:53.811581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.811560 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Oct 29 23:12:53.811698 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.811572 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Oct 29 23:12:53.811822 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.811730 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w5p5k\"" Oct 29 23:12:53.811822 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.811794 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Oct 29 23:12:53.812275 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.812238 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vktqh\"" Oct 29 23:12:53.812436 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.812382 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-z28mk\"" Oct 29 23:12:53.812691 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.812673 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 29 23:12:53.812886 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.812873 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-sw5tb\"" Oct 29 23:12:53.820011 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.819992 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 29 23:12:53.820096 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.819991 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Oct 29 23:12:53.820096 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.820024 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-b5rdd\"" Oct 29 23:12:53.820096 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.820016 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Oct 29 23:12:53.820221 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.820123 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 29 23:12:53.820221 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.820192 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Oct 29 23:12:53.831903 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.831882 5056 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.832004 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.831976 5056 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.832993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.832978 5056 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 29 23:12:53.856236 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.856214 5056 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 29 23:12:53.856371 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:53.856257 5056 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-130-86.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.858970 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.858943 5056 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 29 23:12:53.859061 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:53.858989 5056 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-86.ec2.internal" Oct 29 23:12:53.890249 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890224 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-run-netns\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.890368 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890272 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-b9jng\" (UID: \"b218e5d9-83c0-4976-90da-471e6b86d460\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" Oct 29 23:12:53.890368 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890355 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-config-volume\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:53.890456 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890381 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-cni-netd\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.890456 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890403 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czvnq\" (UniqueName: \"kubernetes.io/projected/8f11a401-ced0-47dc-8814-c23849ba794b-kube-api-access-czvnq\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mc79c\" (UID: \"8f11a401-ced0-47dc-8814-c23849ba794b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" Oct 29 23:12:53.890456 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890419 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-modprobe-d\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:53.890603 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890492 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-systemd\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:53.890603 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890525 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzc9p\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-kube-api-access-bzc9p\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:53.890603 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890556 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-etc-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.890704 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890611 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/947dc777-7dd0-4244-913a-5dd068e6e6a3-serving-cert\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:53.890704 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890653 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-config-volume\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:53.890779 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890704 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1211641d-1d2c-4de0-b993-5f1e84edc09f-hosts-file\") pod \"node-resolver-zr8dl\" (UID: \"1211641d-1d2c-4de0-b993-5f1e84edc09f\") " pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:53.890779 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890763 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-sys-fs\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:53.890879 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890861 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-cni-bin\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.890879 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890861 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/947dc777-7dd0-4244-913a-5dd068e6e6a3-serving-cert\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:53.890968 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890910 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysconfig\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:53.891021 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890966 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf41b924-24b7-4a89-9d11-af8b4246fcde-dbus\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:53.891021 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.890994 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fce92e88-114c-49c7-b658-193d508126e9-archives-path\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:53.891084 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891041 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-image-registry-private-configuration\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:53.891135 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891081 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-cni-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.891198 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891167 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fce92e88-114c-49c7-b658-193d508126e9-archives-path\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:53.891198 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891177 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-var-lib-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.891299 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891202 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.891299 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891228 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-image-registry-private-configuration\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:53.891299 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891251 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysctl-d\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:53.891299 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891281 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-daemon-config\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.891447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891311 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3b53bf9-0305-43ad-99b1-26f35c841020-data-volume\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:53.891447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891341 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rdlx\" (UniqueName: \"kubernetes.io/projected/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-api-access-9rdlx\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:53.891447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891366 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlhz5\" (UniqueName: \"kubernetes.io/projected/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-kube-api-access-jlhz5\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:53.891447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891390 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/25afb96c-6351-470d-8dfa-dd8a9145be40-ca-trust-extracted\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:53.891447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891412 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3b53bf9-0305-43ad-99b1-26f35c841020-data-volume\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:53.891447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891415 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6eccdb56-305f-4a78-b02b-e3f983030542-host-slash\") pod \"iptables-alerter-c9pz5\" (UID: \"6eccdb56-305f-4a78-b02b-e3f983030542\") " pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:53.891729 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891467 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2f44\" (UniqueName: \"kubernetes.io/projected/ad5dcaa4-c226-4639-92fe-ace864d049ac-kube-api-access-g2f44\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:53.891729 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891494 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/31ddd1aa-9d35-47bd-9553-9538f3517707-signing-key\") pod \"service-ca-5fffc97bdf-t9kdp\" (UID: \"31ddd1aa-9d35-47bd-9553-9538f3517707\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" Oct 29 23:12:53.891729 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891535 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/25afb96c-6351-470d-8dfa-dd8a9145be40-ca-trust-extracted\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:53.891729 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891630 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-daemon-config\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.891729 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891668 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:53.891729 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891713 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-host\") pod \"node-ca-jg249\" (UID: \"c652cee1-24f4-460a-8e1a-669b0ae9c0b1\") " pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:53.891984 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891757 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-kubelet\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.891984 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891761 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/31ddd1aa-9d35-47bd-9553-9538f3517707-signing-key\") pod \"service-ca-5fffc97bdf-t9kdp\" (UID: \"31ddd1aa-9d35-47bd-9553-9538f3517707\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" Oct 29 23:12:53.891984 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891785 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a1c203ed-13a2-429b-8a07-34751a25ca06-ovn-node-metrics-cert\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.891984 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891804 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-lib-modules\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:53.891984 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891837 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-hostroot\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.891984 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891936 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/837e8971-01d7-44f5-81bb-bf206d191f2d-serving-cert\") pod \"service-ca-operator-544d749c8c-b9jwd\" (UID: \"837e8971-01d7-44f5-81bb-bf206d191f2d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" Oct 29 23:12:53.891984 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891969 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad5dcaa4-c226-4639-92fe-ace864d049ac-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:53.892283 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.891998 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:53.892283 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892121 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/837e8971-01d7-44f5-81bb-bf206d191f2d-serving-cert\") pod \"service-ca-operator-544d749c8c-b9jwd\" (UID: \"837e8971-01d7-44f5-81bb-bf206d191f2d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" Oct 29 23:12:53.892283 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892110 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtg4p\" (UniqueName: \"kubernetes.io/projected/ae184786-7bc6-403a-87fc-185baf65ffa5-kube-api-access-vtg4p\") pod \"kas-connection-checker-685895c7c9-mjz4s\" (UID: \"ae184786-7bc6-403a-87fc-185baf65ffa5\") " pod="kube-system/kas-connection-checker-685895c7c9-mjz4s" Oct 29 23:12:53.892283 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892202 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a1c203ed-13a2-429b-8a07-34751a25ca06-ovn-node-metrics-cert\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.892283 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892251 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cnibin\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:53.892472 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892312 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/63784b03-969b-4e2a-b799-e027ad1c0183-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:53.892472 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892327 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad5dcaa4-c226-4639-92fe-ace864d049ac-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:53.892472 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892371 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87lrl\" (UniqueName: \"kubernetes.io/projected/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-kube-api-access-87lrl\") pod \"node-ca-jg249\" (UID: \"c652cee1-24f4-460a-8e1a-669b0ae9c0b1\") " pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:53.892472 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892454 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-trusted-ca\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:53.892622 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892486 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsr2l\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-kube-api-access-qsr2l\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:53.892622 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892518 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-system-cni-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.892622 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892542 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/63784b03-969b-4e2a-b799-e027ad1c0183-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:53.892622 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892584 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-socket-dir-parent\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.892622 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892600 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-cni-multus\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.892855 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892696 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1211641d-1d2c-4de0-b993-5f1e84edc09f-tmp-dir\") pod \"node-resolver-zr8dl\" (UID: \"1211641d-1d2c-4de0-b993-5f1e84edc09f\") " pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:53.892855 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892718 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f11a401-ced0-47dc-8814-c23849ba794b-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mc79c\" (UID: \"8f11a401-ced0-47dc-8814-c23849ba794b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" Oct 29 23:12:53.892855 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892736 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-run\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:53.892855 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892788 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqrk2\" (UniqueName: \"kubernetes.io/projected/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-kube-api-access-xqrk2\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:53.892855 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892833 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh8rg\" (UniqueName: \"kubernetes.io/projected/1211641d-1d2c-4de0-b993-5f1e84edc09f-kube-api-access-gh8rg\") pod \"node-resolver-zr8dl\" (UID: \"1211641d-1d2c-4de0-b993-5f1e84edc09f\") " pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:53.892855 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892841 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1211641d-1d2c-4de0-b993-5f1e84edc09f-tmp-dir\") pod \"node-resolver-zr8dl\" (UID: \"1211641d-1d2c-4de0-b993-5f1e84edc09f\") " pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:53.893130 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892859 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-certificates\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:53.893130 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892884 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8f11a401-ced0-47dc-8814-c23849ba794b-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mc79c\" (UID: \"8f11a401-ced0-47dc-8814-c23849ba794b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" Oct 29 23:12:53.893130 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.892974 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-etc-kubernetes\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.893130 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893082 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-env-overrides\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.893130 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893103 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-default-certificate\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:53.893418 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893140 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chhmv\" (UniqueName: \"kubernetes.io/projected/31ddd1aa-9d35-47bd-9553-9538f3517707-kube-api-access-chhmv\") pod \"service-ca-5fffc97bdf-t9kdp\" (UID: \"31ddd1aa-9d35-47bd-9553-9538f3517707\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" Oct 29 23:12:53.893418 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893157 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:53.893418 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893183 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/947dc777-7dd0-4244-913a-5dd068e6e6a3-config\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:53.893418 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893358 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-env-overrides\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.893418 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893396 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b218e5d9-83c0-4976-90da-471e6b86d460-nginx-conf\") pod \"networking-console-plugin-86c478bb88-b9jng\" (UID: \"b218e5d9-83c0-4976-90da-471e6b86d460\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" Oct 29 23:12:53.893651 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893420 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-trusted-ca\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:53.893651 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893427 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnjnb\" (UniqueName: \"kubernetes.io/projected/1575b8c7-d3a3-4956-bb56-72ba26dae4f6-kube-api-access-fnjnb\") pod \"network-check-source-6f78d59f99-nhgxp\" (UID: \"1575b8c7-d3a3-4956-bb56-72ba26dae4f6\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nhgxp" Oct 29 23:12:53.893651 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893477 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-installation-pull-secrets\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:53.893651 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893483 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:53.893651 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893510 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-image-registry-private-configuration\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:53.893651 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893565 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4748v\" (UniqueName: \"kubernetes.io/projected/a543490c-5c05-4468-a0ad-25ebb475e78e-kube-api-access-4748v\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.893651 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893590 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-var-lib-kubelet\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:53.893651 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893603 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/947dc777-7dd0-4244-913a-5dd068e6e6a3-config\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:53.893651 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893615 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-tuned\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:53.894052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893694 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-default-certificate\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:53.894052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893700 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-certificates\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:53.894052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893712 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4tx9\" (UniqueName: \"kubernetes.io/projected/837e8971-01d7-44f5-81bb-bf206d191f2d-kube-api-access-m4tx9\") pod \"service-ca-operator-544d749c8c-b9jwd\" (UID: \"837e8971-01d7-44f5-81bb-bf206d191f2d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" Oct 29 23:12:53.894052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893746 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-stats-auth\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:53.894052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893750 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-tuned\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:53.894052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893767 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-installation-pull-secrets\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:53.894052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893857 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-image-registry-private-configuration\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:53.894052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893895 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-socket-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:53.894052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893905 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/b218e5d9-83c0-4976-90da-471e6b86d460-nginx-conf\") pod \"networking-console-plugin-86c478bb88-b9jng\" (UID: \"b218e5d9-83c0-4976-90da-471e6b86d460\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" Oct 29 23:12:53.894052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893939 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf5x4\" (UniqueName: \"kubernetes.io/projected/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-kube-api-access-qf5x4\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:53.894052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893983 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-stats-auth\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:53.894052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.893975 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-system-cni-dir\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:53.894052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894036 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/837e8971-01d7-44f5-81bb-bf206d191f2d-config\") pod \"service-ca-operator-544d749c8c-b9jwd\" (UID: \"837e8971-01d7-44f5-81bb-bf206d191f2d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" Oct 29 23:12:53.894581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894071 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:53.894581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894120 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-multus-certs\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.894581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894148 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-sys\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:53.894581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894188 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/837e8971-01d7-44f5-81bb-bf206d191f2d-config\") pod \"service-ca-operator-544d749c8c-b9jwd\" (UID: \"837e8971-01d7-44f5-81bb-bf206d191f2d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" Oct 29 23:12:53.894581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894208 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:53.894581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894238 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-ovnkube-config\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.894581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894283 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf41b924-24b7-4a89-9d11-af8b4246fcde-kubelet-config\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:53.894581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894320 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwkv6\" (UniqueName: \"kubernetes.io/projected/63784b03-969b-4e2a-b799-e027ad1c0183-kube-api-access-zwkv6\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:53.894581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894391 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:53.894581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894422 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-ovnkube-script-lib\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.894581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894483 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc28v\" (UniqueName: \"kubernetes.io/projected/7153f183-6e65-4aed-b021-d0a8c3070fce-kube-api-access-hc28v\") pod \"migrator-66df5d496b-75lxf\" (UID: \"7153f183-6e65-4aed-b021-d0a8c3070fce\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-75lxf" Oct 29 23:12:53.894581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894515 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:53.894581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894530 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-ovnkube-config\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.894581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894544 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-trusted-ca\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:53.894581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894585 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-cni-bin\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894608 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-ovn\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894633 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysctl-conf\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894699 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a1c203ed-13a2-429b-8a07-34751a25ca06-ovnkube-script-lib\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894750 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfhfl\" (UniqueName: \"kubernetes.io/projected/21385963-6be4-4e3f-a996-f48b0c9ab0b1-kube-api-access-jfhfl\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894795 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894834 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ad5dcaa4-c226-4639-92fe-ace864d049ac-serving-cert\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894863 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shjmc\" (UniqueName: \"kubernetes.io/projected/fce92e88-114c-49c7-b658-193d508126e9-kube-api-access-shjmc\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894887 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqgl9\" (UniqueName: \"kubernetes.io/projected/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-kube-api-access-vqgl9\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894911 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6eccdb56-305f-4a78-b02b-e3f983030542-iptables-alerter-script\") pod \"iptables-alerter-c9pz5\" (UID: \"6eccdb56-305f-4a78-b02b-e3f983030542\") " pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894941 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-installation-pull-secrets\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894966 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rftvj\" (UniqueName: \"kubernetes.io/projected/fa19a031-f171-495b-a72d-f4f79b88fceb-kube-api-access-rftvj\") pod \"ingress-canary-t8vj4\" (UID: \"fa19a031-f171-495b-a72d-f4f79b88fceb\") " pod="openshift-ingress-canary/ingress-canary-t8vj4" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894968 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-trusted-ca\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.894991 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895011 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ad5dcaa4-c226-4639-92fe-ace864d049ac-serving-cert\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895023 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-trusted-ca-bundle\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895065 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895098 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-registration-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895122 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-ca-trust-extracted\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895123 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cf41b924-24b7-4a89-9d11-af8b4246fcde-original-pull-secret\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895144 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa19a031-f171-495b-a72d-f4f79b88fceb-cert\") pod \"ingress-canary-t8vj4\" (UID: \"fa19a031-f171-495b-a72d-f4f79b88fceb\") " pod="openshift-ingress-canary/ingress-canary-t8vj4" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895186 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-kubernetes\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895218 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-tmp\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:53.895233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895246 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:53.896052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895283 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-ca-trust-extracted\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:53.896052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895297 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-bound-sa-token\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:53.896052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895232 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:53.896052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895335 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f11a401-ced0-47dc-8814-c23849ba794b-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mc79c\" (UID: \"8f11a401-ced0-47dc-8814-c23849ba794b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" Oct 29 23:12:53.896052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895355 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6eccdb56-305f-4a78-b02b-e3f983030542-iptables-alerter-script\") pod \"iptables-alerter-c9pz5\" (UID: \"6eccdb56-305f-4a78-b02b-e3f983030542\") " pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:53.896052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895429 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-installation-pull-secrets\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:53.896052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895484 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-tmp\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:53.896052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895493 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ad5dcaa4-c226-4639-92fe-ace864d049ac-snapshots\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:53.896052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895539 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fa19a031-f171-495b-a72d-f4f79b88fceb-cert\") pod \"ingress-canary-t8vj4\" (UID: \"fa19a031-f171-495b-a72d-f4f79b88fceb\") " pod="openshift-ingress-canary/ingress-canary-t8vj4" Oct 29 23:12:53.896052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895539 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8f11a401-ced0-47dc-8814-c23849ba794b-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mc79c\" (UID: \"8f11a401-ced0-47dc-8814-c23849ba794b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" Oct 29 23:12:53.896052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895609 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-877j7\" (UniqueName: \"kubernetes.io/projected/5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd-kube-api-access-877j7\") pod \"kas-connection-checker-685895c7c9-z6qn8\" (UID: \"5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd\") " pod="kube-system/kas-connection-checker-685895c7c9-z6qn8" Oct 29 23:12:53.896052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895726 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/ad5dcaa4-c226-4639-92fe-ace864d049ac-snapshots\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:53.896052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895733 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-trusted-ca-bundle\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:53.896052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895764 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/947dc777-7dd0-4244-913a-5dd068e6e6a3-trusted-ca\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:53.896052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895796 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:53.896052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.895823 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad5dcaa4-c226-4639-92fe-ace864d049ac-service-ca-bundle\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:53.896648 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896087 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/947dc777-7dd0-4244-913a-5dd068e6e6a3-trusted-ca\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:53.896648 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896198 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad5dcaa4-c226-4639-92fe-ace864d049ac-service-ca-bundle\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:53.896648 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896231 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4mr2\" (UniqueName: \"kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2\") pod \"network-check-target-dn96d\" (UID: \"b368f5ac-f18a-49d2-b9be-6cca37b7505b\") " pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:53.896648 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896243 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:53.896648 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896284 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-certificates\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:53.896648 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896405 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-systemd-units\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.896648 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896456 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dpmm\" (UniqueName: \"kubernetes.io/projected/947dc777-7dd0-4244-913a-5dd068e6e6a3-kube-api-access-9dpmm\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:53.896648 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896498 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-etc-selinux\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:53.896648 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896530 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-bound-sa-token\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:53.896648 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896565 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-host\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:53.896648 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896622 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-kubelet\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.897100 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896677 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b3b53bf9-0305-43ad-99b1-26f35c841020-crio-socket\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:53.897100 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896717 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:53.897100 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896766 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-serviceca\") pod \"node-ca-jg249\" (UID: \"c652cee1-24f4-460a-8e1a-669b0ae9c0b1\") " pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:53.897100 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896811 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r5qh\" (UniqueName: \"kubernetes.io/projected/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-kube-api-access-4r5qh\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:53.897100 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896929 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-k8s-cni-cncf-io\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.897100 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896968 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-node-log\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.897100 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.896989 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-certificates\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:53.897100 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897033 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:53.897100 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897072 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swjdw\" (UniqueName: \"kubernetes.io/projected/bf94acd7-605c-4d9a-9021-5ce66a126edd-kube-api-access-swjdw\") pod \"kas-connection-checker-685895c7c9-74vqh\" (UID: \"bf94acd7-605c-4d9a-9021-5ce66a126edd\") " pod="kube-system/kas-connection-checker-685895c7c9-74vqh" Oct 29 23:12:53.897487 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897120 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-service-ca-bundle\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:53.897487 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897154 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-conf-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.897487 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897207 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:53.897487 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897236 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b24kn\" (UniqueName: \"kubernetes.io/projected/07425f46-dc9b-4d82-a1b8-246162054ea9-kube-api-access-b24kn\") pod \"migrator-66df5d496b-j6lj7\" (UID: \"07425f46-dc9b-4d82-a1b8-246162054ea9\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j6lj7" Oct 29 23:12:53.897487 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897277 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad5dcaa4-c226-4639-92fe-ace864d049ac-tmp\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:53.897487 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897324 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-systemd\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.897487 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897359 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7sk4\" (UniqueName: \"kubernetes.io/projected/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-kube-api-access-m7sk4\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:53.897487 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897360 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-service-ca-bundle\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:53.897487 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897394 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:53.897487 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897439 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-serviceca\") pod \"node-ca-jg249\" (UID: \"c652cee1-24f4-460a-8e1a-669b0ae9c0b1\") " pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:53.897853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897505 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-cnibin\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.897853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897546 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad5dcaa4-c226-4639-92fe-ace864d049ac-tmp\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:53.897853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897609 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-slash\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.897853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897615 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:53.897853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897661 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-log-socket\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.897853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897690 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-run-ovn-kubernetes\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.897853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897708 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhszk\" (UniqueName: \"kubernetes.io/projected/a1c203ed-13a2-429b-8a07-34751a25ca06-kube-api-access-xhszk\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.897853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897759 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrdbd\" (UniqueName: \"kubernetes.io/projected/350f9704-1e0e-4f41-958b-8b0b16761375-kube-api-access-rrdbd\") pod \"volume-data-source-validator-58c9f5f46d-pn894\" (UID: \"350f9704-1e0e-4f41-958b-8b0b16761375\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pn894" Oct 29 23:12:53.898231 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897919 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c964bbd5-fa82-459f-b464-d27fd1080654-agent-certs\") pod \"konnectivity-agent-25m4p\" (UID: \"c964bbd5-fa82-459f-b464-d27fd1080654\") " pod="kube-system/konnectivity-agent-25m4p" Oct 29 23:12:53.898231 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897940 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-tmp-dir\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:53.898231 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897956 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbjw5\" (UniqueName: \"kubernetes.io/projected/6eccdb56-305f-4a78-b02b-e3f983030542-kube-api-access-nbjw5\") pod \"iptables-alerter-c9pz5\" (UID: \"6eccdb56-305f-4a78-b02b-e3f983030542\") " pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:53.898231 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897974 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-device-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:53.898231 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.897996 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a543490c-5c05-4468-a0ad-25ebb475e78e-cni-binary-copy\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.898231 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.898017 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-tmp-dir\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:53.898231 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.898036 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-netns\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.898231 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.898070 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.898231 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.898098 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/31ddd1aa-9d35-47bd-9553-9538f3517707-signing-cabundle\") pod \"service-ca-5fffc97bdf-t9kdp\" (UID: \"31ddd1aa-9d35-47bd-9553-9538f3517707\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" Oct 29 23:12:53.898231 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.898127 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c964bbd5-fa82-459f-b464-d27fd1080654-konnectivity-ca\") pod \"konnectivity-agent-25m4p\" (UID: \"c964bbd5-fa82-459f-b464-d27fd1080654\") " pod="kube-system/konnectivity-agent-25m4p" Oct 29 23:12:53.898231 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.898158 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:53.898618 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.898282 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c964bbd5-fa82-459f-b464-d27fd1080654-agent-certs\") pod \"konnectivity-agent-25m4p\" (UID: \"c964bbd5-fa82-459f-b464-d27fd1080654\") " pod="kube-system/konnectivity-agent-25m4p" Oct 29 23:12:53.898618 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.898292 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c964bbd5-fa82-459f-b464-d27fd1080654-konnectivity-ca\") pod \"konnectivity-agent-25m4p\" (UID: \"c964bbd5-fa82-459f-b464-d27fd1080654\") " pod="kube-system/konnectivity-agent-25m4p" Oct 29 23:12:53.898618 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.898316 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a543490c-5c05-4468-a0ad-25ebb475e78e-cni-binary-copy\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.898618 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.898439 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/31ddd1aa-9d35-47bd-9553-9538f3517707-signing-cabundle\") pod \"service-ca-5fffc97bdf-t9kdp\" (UID: \"31ddd1aa-9d35-47bd-9553-9538f3517707\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" Oct 29 23:12:53.912700 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.912675 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czvnq\" (UniqueName: \"kubernetes.io/projected/8f11a401-ced0-47dc-8814-c23849ba794b-kube-api-access-czvnq\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-mc79c\" (UID: \"8f11a401-ced0-47dc-8814-c23849ba794b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" Oct 29 23:12:53.912853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.912835 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzc9p\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-kube-api-access-bzc9p\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:53.913131 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.913103 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rdlx\" (UniqueName: \"kubernetes.io/projected/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-api-access-9rdlx\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:53.913375 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.913357 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlhz5\" (UniqueName: \"kubernetes.io/projected/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-kube-api-access-jlhz5\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:53.919174 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.919154 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtg4p\" (UniqueName: \"kubernetes.io/projected/ae184786-7bc6-403a-87fc-185baf65ffa5-kube-api-access-vtg4p\") pod \"kas-connection-checker-685895c7c9-mjz4s\" (UID: \"ae184786-7bc6-403a-87fc-185baf65ffa5\") " pod="kube-system/kas-connection-checker-685895c7c9-mjz4s" Oct 29 23:12:53.919296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.919226 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87lrl\" (UniqueName: \"kubernetes.io/projected/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-kube-api-access-87lrl\") pod \"node-ca-jg249\" (UID: \"c652cee1-24f4-460a-8e1a-669b0ae9c0b1\") " pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:53.919560 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.919539 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsr2l\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-kube-api-access-qsr2l\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:53.919620 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.919554 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2f44\" (UniqueName: \"kubernetes.io/projected/ad5dcaa4-c226-4639-92fe-ace864d049ac-kube-api-access-g2f44\") pod \"insights-operator-7c4986d449-m72dz\" (UID: \"ad5dcaa4-c226-4639-92fe-ace864d049ac\") " pod="openshift-insights/insights-operator-7c4986d449-m72dz" Oct 29 23:12:53.928688 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.928218 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4748v\" (UniqueName: \"kubernetes.io/projected/a543490c-5c05-4468-a0ad-25ebb475e78e-kube-api-access-4748v\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.928688 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.928322 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf5x4\" (UniqueName: \"kubernetes.io/projected/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-kube-api-access-qf5x4\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:53.928688 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.928353 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4tx9\" (UniqueName: \"kubernetes.io/projected/837e8971-01d7-44f5-81bb-bf206d191f2d-kube-api-access-m4tx9\") pod \"service-ca-operator-544d749c8c-b9jwd\" (UID: \"837e8971-01d7-44f5-81bb-bf206d191f2d\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" Oct 29 23:12:53.928688 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.928426 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnjnb\" (UniqueName: \"kubernetes.io/projected/1575b8c7-d3a3-4956-bb56-72ba26dae4f6-kube-api-access-fnjnb\") pod \"network-check-source-6f78d59f99-nhgxp\" (UID: \"1575b8c7-d3a3-4956-bb56-72ba26dae4f6\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-nhgxp" Oct 29 23:12:53.928688 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.928648 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqrk2\" (UniqueName: \"kubernetes.io/projected/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-kube-api-access-xqrk2\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:53.928960 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.928694 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh8rg\" (UniqueName: \"kubernetes.io/projected/1211641d-1d2c-4de0-b993-5f1e84edc09f-kube-api-access-gh8rg\") pod \"node-resolver-zr8dl\" (UID: \"1211641d-1d2c-4de0-b993-5f1e84edc09f\") " pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:53.929391 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.929367 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-bound-sa-token\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:53.929493 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.929417 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chhmv\" (UniqueName: \"kubernetes.io/projected/31ddd1aa-9d35-47bd-9553-9538f3517707-kube-api-access-chhmv\") pod \"service-ca-5fffc97bdf-t9kdp\" (UID: \"31ddd1aa-9d35-47bd-9553-9538f3517707\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t9kdp" Oct 29 23:12:53.929549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.929524 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dpmm\" (UniqueName: \"kubernetes.io/projected/947dc777-7dd0-4244-913a-5dd068e6e6a3-kube-api-access-9dpmm\") pod \"console-operator-7cd56b8dbc-2gzh7\" (UID: \"947dc777-7dd0-4244-913a-5dd068e6e6a3\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:53.929549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.929531 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r5qh\" (UniqueName: \"kubernetes.io/projected/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-kube-api-access-4r5qh\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:53.932727 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.932709 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rftvj\" (UniqueName: \"kubernetes.io/projected/fa19a031-f171-495b-a72d-f4f79b88fceb-kube-api-access-rftvj\") pod \"ingress-canary-t8vj4\" (UID: \"fa19a031-f171-495b-a72d-f4f79b88fceb\") " pod="openshift-ingress-canary/ingress-canary-t8vj4" Oct 29 23:12:53.933414 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.933391 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbjw5\" (UniqueName: \"kubernetes.io/projected/6eccdb56-305f-4a78-b02b-e3f983030542-kube-api-access-nbjw5\") pod \"iptables-alerter-c9pz5\" (UID: \"6eccdb56-305f-4a78-b02b-e3f983030542\") " pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:53.933512 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.933412 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqgl9\" (UniqueName: \"kubernetes.io/projected/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-kube-api-access-vqgl9\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:53.933512 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.933501 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhszk\" (UniqueName: \"kubernetes.io/projected/a1c203ed-13a2-429b-8a07-34751a25ca06-kube-api-access-xhszk\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.933636 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.933619 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7sk4\" (UniqueName: \"kubernetes.io/projected/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-kube-api-access-m7sk4\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:53.933835 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.933817 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4mr2\" (UniqueName: \"kubernetes.io/projected/b368f5ac-f18a-49d2-b9be-6cca37b7505b-kube-api-access-t4mr2\") pod \"network-check-target-dn96d\" (UID: \"b368f5ac-f18a-49d2-b9be-6cca37b7505b\") " pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:53.934214 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.934199 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b24kn\" (UniqueName: \"kubernetes.io/projected/07425f46-dc9b-4d82-a1b8-246162054ea9-kube-api-access-b24kn\") pod \"migrator-66df5d496b-j6lj7\" (UID: \"07425f46-dc9b-4d82-a1b8-246162054ea9\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j6lj7" Oct 29 23:12:53.934327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.934306 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swjdw\" (UniqueName: \"kubernetes.io/projected/bf94acd7-605c-4d9a-9021-5ce66a126edd-kube-api-access-swjdw\") pod \"kas-connection-checker-685895c7c9-74vqh\" (UID: \"bf94acd7-605c-4d9a-9021-5ce66a126edd\") " pod="kube-system/kas-connection-checker-685895c7c9-74vqh" Oct 29 23:12:53.934488 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.934474 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shjmc\" (UniqueName: \"kubernetes.io/projected/fce92e88-114c-49c7-b658-193d508126e9-kube-api-access-shjmc\") pod \"periodic-gathering-hhksv-vlwlb\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:12:53.934565 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.934543 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwkv6\" (UniqueName: \"kubernetes.io/projected/63784b03-969b-4e2a-b799-e027ad1c0183-kube-api-access-zwkv6\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:53.934635 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.934620 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrdbd\" (UniqueName: \"kubernetes.io/projected/350f9704-1e0e-4f41-958b-8b0b16761375-kube-api-access-rrdbd\") pod \"volume-data-source-validator-58c9f5f46d-pn894\" (UID: \"350f9704-1e0e-4f41-958b-8b0b16761375\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pn894" Oct 29 23:12:53.935452 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.935434 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-877j7\" (UniqueName: \"kubernetes.io/projected/5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd-kube-api-access-877j7\") pod \"kas-connection-checker-685895c7c9-z6qn8\" (UID: \"5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd\") " pod="kube-system/kas-connection-checker-685895c7c9-z6qn8" Oct 29 23:12:53.937733 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.937714 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-bound-sa-token\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:53.938439 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.938424 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc28v\" (UniqueName: \"kubernetes.io/projected/7153f183-6e65-4aed-b021-d0a8c3070fce-kube-api-access-hc28v\") pod \"migrator-66df5d496b-75lxf\" (UID: \"7153f183-6e65-4aed-b021-d0a8c3070fce\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-75lxf" Oct 29 23:12:53.938498 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.938458 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfhfl\" (UniqueName: \"kubernetes.io/projected/21385963-6be4-4e3f-a996-f48b0c9ab0b1-kube-api-access-jfhfl\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:53.999532 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999475 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-conf-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.999532 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999504 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-systemd\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.999532 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999528 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:53.999693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999542 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-cnibin\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.999693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999558 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-slash\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.999693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999577 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-log-socket\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.999693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999599 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-run-ovn-kubernetes\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.999693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999612 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-slash\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.999693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999625 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-log-socket\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.999693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999632 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-run-ovn-kubernetes\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.999693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999631 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-cnibin\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.999693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999576 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-conf-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:53.999693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999666 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-device-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:53.999693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999598 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-systemd\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:53.999693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999628 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-device-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999732 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-netns\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999759 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999781 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999791 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-netns\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999799 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999805 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-run-netns\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999829 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-b9jng\" (UID: \"b218e5d9-83c0-4976-90da-471e6b86d460\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999833 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999861 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-cni-netd\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999868 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-run-netns\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999886 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-modprobe-d\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999890 5056 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999920 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-systemd\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999948 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-cni-netd\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999957 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-etc-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999981 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1211641d-1d2c-4de0-b993-5f1e84edc09f-hosts-file\") pod \"node-resolver-zr8dl\" (UID: \"1211641d-1d2c-4de0-b993-5f1e84edc09f\") " pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:53.999995 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-modprobe-d\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000005 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-sys-fs\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000013 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-etc-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000027 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-cni-bin\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000043 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1211641d-1d2c-4de0-b993-5f1e84edc09f-hosts-file\") pod \"node-resolver-zr8dl\" (UID: \"1211641d-1d2c-4de0-b993-5f1e84edc09f\") " pod="openshift-dns/node-resolver-zr8dl" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000047 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysconfig\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000057 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-sys-fs\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000058 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-cni-bin\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000013 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-systemd\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000080 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf41b924-24b7-4a89-9d11-af8b4246fcde-dbus\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000108 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysconfig\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.000097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000113 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-cni-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000129 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cf41b924-24b7-4a89-9d11-af8b4246fcde-dbus\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000147 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-var-lib-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000168 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000168 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-cni-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000200 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000200 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-var-lib-openvswitch\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000224 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysctl-d\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000280 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6eccdb56-305f-4a78-b02b-e3f983030542-host-slash\") pod \"iptables-alerter-c9pz5\" (UID: \"6eccdb56-305f-4a78-b02b-e3f983030542\") " pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000309 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000325 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6eccdb56-305f-4a78-b02b-e3f983030542-host-slash\") pod \"iptables-alerter-c9pz5\" (UID: \"6eccdb56-305f-4a78-b02b-e3f983030542\") " pod="openshift-network-operator/iptables-alerter-c9pz5" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000332 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-host\") pod \"node-ca-jg249\" (UID: \"c652cee1-24f4-460a-8e1a-669b0ae9c0b1\") " pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000362 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-kubelet\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000367 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c652cee1-24f4-460a-8e1a-669b0ae9c0b1-host\") pod \"node-ca-jg249\" (UID: \"c652cee1-24f4-460a-8e1a-669b0ae9c0b1\") " pod="openshift-image-registry/node-ca-jg249" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000374 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysctl-d\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000387 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-lib-modules\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000409 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-host-kubelet\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000411 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-hostroot\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000443 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-hostroot\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000443 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000490 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cnibin\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000522 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-system-cni-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.000529 5056 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000545 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-lib-modules\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000549 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-socket-dir-parent\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000580 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-cni-multus\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000606 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-run\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000618 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-system-cni-dir\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000631 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-etc-kubernetes\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.000660 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls podName:63784b03-969b-4e2a-b799-e027ad1c0183 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:54.500640898 +0000 UTC m=+1.867388556 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-t6gcj" (UID: "63784b03-969b-4e2a-b799-e027ad1c0183") : secret "cluster-monitoring-operator-tls" not found Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000669 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-etc-kubernetes\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000697 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-var-lib-kubelet\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000723 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-socket-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000582 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-cnibin\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000750 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-system-cni-dir\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000753 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-cni-multus\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000784 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000794 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-run\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000805 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-var-lib-kubelet\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000810 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-multus-certs\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000832 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-system-cni-dir\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000837 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-sys\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000854 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-socket-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000724 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-multus-socket-dir-parent\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000871 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000879 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-sys\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000901 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf41b924-24b7-4a89-9d11-af8b4246fcde-kubelet-config\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000928 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.000951 5056 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.000982 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.000996 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls podName:8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:54.500981359 +0000 UTC m=+1.867729007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-jfvvm" (UID: "8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4") : secret "samples-operator-tls" not found Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001025 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-cni-bin\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001047 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-ovn\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001075 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysctl-conf\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001110 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001113 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-cni-bin\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001147 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-registration-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001168 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-multus-certs\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001187 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-kubernetes\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.001179 5056 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001214 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-run-ovn\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001220 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.001292 5056 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.001297 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs podName:c6fc8782-2d71-4cb3-ab4b-b1d14aad460f nodeName:}" failed. No retries permitted until 2026-10-29 23:12:54.501256465 +0000 UTC m=+1.868004098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs") pod "network-metrics-daemon-2xktq" (UID: "c6fc8782-2d71-4cb3-ab4b-b1d14aad460f") : secret "metrics-daemon-secret" not found Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001379 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cf41b924-24b7-4a89-9d11-af8b4246fcde-kubelet-config\") pod \"global-pull-secret-syncer-gz4mg\" (UID: \"cf41b924-24b7-4a89-9d11-af8b4246fcde\") " pod="kube-system/global-pull-secret-syncer-gz4mg" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001395 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-sysctl-conf\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.001415 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm podName:b3b53bf9-0305-43ad-99b1-26f35c841020 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:54.501398398 +0000 UTC m=+1.868146032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x975f" (UID: "b3b53bf9-0305-43ad-99b1-26f35c841020") : configmap "kube-rbac-proxy" not found Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001441 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-systemd-units\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.001423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001468 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-etc-selinux\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001492 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-host\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001538 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-kubelet\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001502 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-systemd-units\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001550 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-host\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001537 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-etc-selinux\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001469 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a-etc-kubernetes\") pod \"tuned-wd9cn\" (UID: \"02e4b65a-b1c0-4ed0-9e62-2a9a428f2f5a\") " pod="openshift-cluster-node-tuning-operator/tuned-wd9cn" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001589 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-var-lib-kubelet\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001600 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7ffa4cb8-b390-49d2-839d-3cf43397eb6f-registration-dir\") pod \"aws-ebs-csi-driver-node-mcwfw\" (UID: \"7ffa4cb8-b390-49d2-839d-3cf43397eb6f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mcwfw" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001621 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b3b53bf9-0305-43ad-99b1-26f35c841020-crio-socket\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001073 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-service-ca-bundle\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001655 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001681 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b3b53bf9-0305-43ad-99b1-26f35c841020-crio-socket\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001684 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-k8s-cni-cncf-io\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001711 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-node-log\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001741 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001748 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/a543490c-5c05-4468-a0ad-25ebb475e78e-host-run-k8s-cni-cncf-io\") pod \"multus-49s6q\" (UID: \"a543490c-5c05-4468-a0ad-25ebb475e78e\") " pod="openshift-multus/multus-49s6q" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001766 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a1c203ed-13a2-429b-8a07-34751a25ca06-node-log\") pod \"ovnkube-node-blg76\" (UID: \"a1c203ed-13a2-429b-8a07-34751a25ca06\") " pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.001813 5056 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.001842 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/21385963-6be4-4e3f-a996-f48b0c9ab0b1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzb6b\" (UID: \"21385963-6be4-4e3f-a996-f48b0c9ab0b1\") " pod="openshift-multus/multus-additional-cni-plugins-xzb6b" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.001900 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls podName:b3b53bf9-0305-43ad-99b1-26f35c841020 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:54.501885881 +0000 UTC m=+1.868633508 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x975f" (UID: "b3b53bf9-0305-43ad-99b1-26f35c841020") : secret "insights-runtime-extractor-tls" not found Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.002703 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b218e5d9-83c0-4976-90da-471e6b86d460-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-b9jng\" (UID: \"b218e5d9-83c0-4976-90da-471e6b86d460\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.003345 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls\") pod \"image-registry-5679dbd569-548pj\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.003372 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls\") pod \"image-registry-7cbdc4b876-9xjlm\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.003896 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/8388282a-8b2c-4f5c-ab3d-ec58ea20cc93-metrics-tls\") pod \"dns-default-gnwhl\" (UID: \"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93\") " pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:54.004579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.004023 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6ad3759-e11c-476d-82ee-fffdf8fd6d47-metrics-certs\") pod \"router-default-866787b6f9-24krt\" (UID: \"b6ad3759-e11c-476d-82ee-fffdf8fd6d47\") " pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:54.022424 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.022398 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:54.062639 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.062602 5056 scope.go:122] "RemoveContainer" containerID="a32678eaae6ae6d7b889b7350cba11dc4d854c7dae6d8bfaa12ca78621468dbb" Oct 29 23:12:54.062792 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.062683 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:54.062792 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.062723 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:54.063252 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.063230 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" Oct 29 23:12:54.075034 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.074947 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:54.076998 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.076974 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:54.182659 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.174354 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:54.233853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.232378 5056 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:54.431838 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.431812 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5679dbd569-548pj"] Oct 29 23:12:54.434550 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:54.434517 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d3d6fb4_8dac_4983_9fa1_84fb5b1b5a6c.slice/crio-cc1263d3065d9d848697d2002e7057f0b4a69cebaeacbb2b3d0b07766c2e7275 WatchSource:0}: Error finding container cc1263d3065d9d848697d2002e7057f0b4a69cebaeacbb2b3d0b07766c2e7275: Status 404 returned error can't find the container with id cc1263d3065d9d848697d2002e7057f0b4a69cebaeacbb2b3d0b07766c2e7275 Oct 29 23:12:54.473774 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.473747 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-866787b6f9-24krt"] Oct 29 23:12:54.476398 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:54.476373 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6ad3759_e11c_476d_82ee_fffdf8fd6d47.slice/crio-0ce937013285b6fb94e25bd4adf1685fcb3fecd62ae0b1d95031dd7fe1530dd7 WatchSource:0}: Error finding container 0ce937013285b6fb94e25bd4adf1685fcb3fecd62ae0b1d95031dd7fe1530dd7: Status 404 returned error can't find the container with id 0ce937013285b6fb94e25bd4adf1685fcb3fecd62ae0b1d95031dd7fe1530dd7 Oct 29 23:12:54.492737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.492711 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gnwhl"] Oct 29 23:12:54.505681 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.505642 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:54.505763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.505710 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:54.505763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.505742 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:54.505862 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.505772 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:54.505862 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.505809 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:54.505985 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.505902 5056 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Oct 29 23:12:54.505985 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.505965 5056 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 29 23:12:54.506076 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.505965 5056 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 29 23:12:54.506076 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.505968 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs podName:c6fc8782-2d71-4cb3-ab4b-b1d14aad460f nodeName:}" failed. No retries permitted until 2026-10-29 23:12:55.50595289 +0000 UTC m=+2.872700504 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs") pod "network-metrics-daemon-2xktq" (UID: "c6fc8782-2d71-4cb3-ab4b-b1d14aad460f") : secret "metrics-daemon-secret" not found Oct 29 23:12:54.506076 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.506059 5056 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 29 23:12:54.506076 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.506076 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls podName:63784b03-969b-4e2a-b799-e027ad1c0183 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:55.506052353 +0000 UTC m=+2.872799965 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-t6gcj" (UID: "63784b03-969b-4e2a-b799-e027ad1c0183") : secret "cluster-monitoring-operator-tls" not found Oct 29 23:12:54.506294 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.506091 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls podName:b3b53bf9-0305-43ad-99b1-26f35c841020 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:55.506083114 +0000 UTC m=+2.872830722 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x975f" (UID: "b3b53bf9-0305-43ad-99b1-26f35c841020") : secret "insights-runtime-extractor-tls" not found Oct 29 23:12:54.506294 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.506112 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm podName:b3b53bf9-0305-43ad-99b1-26f35c841020 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:55.506104977 +0000 UTC m=+2.872852585 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x975f" (UID: "b3b53bf9-0305-43ad-99b1-26f35c841020") : configmap "kube-rbac-proxy" not found Oct 29 23:12:54.506404 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:54.506330 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8388282a_8b2c_4f5c_ab3d_ec58ea20cc93.slice/crio-e3e48b1a3e93a9f9e1aa7288f415b8a2f53f47b0fb81f8216d005dccd18aaeb2 WatchSource:0}: Error finding container e3e48b1a3e93a9f9e1aa7288f415b8a2f53f47b0fb81f8216d005dccd18aaeb2: Status 404 returned error can't find the container with id e3e48b1a3e93a9f9e1aa7288f415b8a2f53f47b0fb81f8216d005dccd18aaeb2 Oct 29 23:12:54.508576 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.508555 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-jfvvm\" (UID: \"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:54.508688 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.508668 5056 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 29 23:12:54.535068 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.535045 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-b9jng"] Oct 29 23:12:54.537868 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:54.537839 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb218e5d9_83c0_4976_90da_471e6b86d460.slice/crio-91693ce912c961cf3f5350c3d33194fa188b2341f22027435586672564159e54 WatchSource:0}: Error finding container 91693ce912c961cf3f5350c3d33194fa188b2341f22027435586672564159e54: Status 404 returned error can't find the container with id 91693ce912c961cf3f5350c3d33194fa188b2341f22027435586672564159e54 Oct 29 23:12:54.540107 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.540087 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cbdc4b876-9xjlm"] Oct 29 23:12:54.542816 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:54.542795 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25afb96c_6351_470d_8dfa_dd8a9145be40.slice/crio-c87ce8d555e9bc090049a675fd0023eb53d730a4c298de5166748ed307fa3802 WatchSource:0}: Error finding container c87ce8d555e9bc090049a675fd0023eb53d730a4c298de5166748ed307fa3802: Status 404 returned error can't find the container with id c87ce8d555e9bc090049a675fd0023eb53d730a4c298de5166748ed307fa3802 Oct 29 23:12:54.659784 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.659750 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" Oct 29 23:12:54.840381 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.840295 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5679dbd569-548pj" event={"ID":"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c","Type":"ContainerStarted","Data":"c8660f151cf84005860e173c5498bfcdb28cb3171a1e49bfcdffd2d0d132964d"} Oct 29 23:12:54.840381 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.840361 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5679dbd569-548pj" event={"ID":"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c","Type":"ContainerStarted","Data":"cc1263d3065d9d848697d2002e7057f0b4a69cebaeacbb2b3d0b07766c2e7275"} Oct 29 23:12:54.841974 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.841934 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/2.log" Oct 29 23:12:54.842474 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.842459 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/1.log" Oct 29 23:12:54.842554 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.842491 5056 generic.go:356] "Generic (PLEG): container finished" podID="947dc777-7dd0-4244-913a-5dd068e6e6a3" containerID="35ddcb9d9abd41d673ad3528ea7c3eaff6e432f5bd52475a5b11ddf923e5a122" exitCode=255 Oct 29 23:12:54.842554 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.842517 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" event={"ID":"947dc777-7dd0-4244-913a-5dd068e6e6a3","Type":"ContainerDied","Data":"35ddcb9d9abd41d673ad3528ea7c3eaff6e432f5bd52475a5b11ddf923e5a122"} Oct 29 23:12:54.842632 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.842571 5056 scope.go:122] "RemoveContainer" containerID="a32678eaae6ae6d7b889b7350cba11dc4d854c7dae6d8bfaa12ca78621468dbb" Oct 29 23:12:54.842824 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.842796 5056 scope.go:122] "RemoveContainer" containerID="35ddcb9d9abd41d673ad3528ea7c3eaff6e432f5bd52475a5b11ddf923e5a122" Oct 29 23:12:54.843097 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:54.843059 5056 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2gzh7_openshift-console-operator(947dc777-7dd0-4244-913a-5dd068e6e6a3)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" podUID="947dc777-7dd0-4244-913a-5dd068e6e6a3" Oct 29 23:12:54.843974 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.843955 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" event={"ID":"b218e5d9-83c0-4976-90da-471e6b86d460","Type":"ContainerStarted","Data":"91693ce912c961cf3f5350c3d33194fa188b2341f22027435586672564159e54"} Oct 29 23:12:54.845233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.845211 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gnwhl" event={"ID":"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93","Type":"ContainerStarted","Data":"e3e48b1a3e93a9f9e1aa7288f415b8a2f53f47b0fb81f8216d005dccd18aaeb2"} Oct 29 23:12:54.846867 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.846849 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-866787b6f9-24krt" event={"ID":"b6ad3759-e11c-476d-82ee-fffdf8fd6d47","Type":"ContainerStarted","Data":"768a3bb80c1aaf6702646ddfc030ed0b67adf00db49a3e0a24c8b90c8a2d113e"} Oct 29 23:12:54.846958 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.846882 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-866787b6f9-24krt" event={"ID":"b6ad3759-e11c-476d-82ee-fffdf8fd6d47","Type":"ContainerStarted","Data":"0ce937013285b6fb94e25bd4adf1685fcb3fecd62ae0b1d95031dd7fe1530dd7"} Oct 29 23:12:54.848571 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.848534 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" event={"ID":"25afb96c-6351-470d-8dfa-dd8a9145be40","Type":"ContainerStarted","Data":"17bd8fa27348541385b4a3db0c1fccaadd53c1bff34b99bf555b8f9c3405c5a1"} Oct 29 23:12:54.848661 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.848574 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" event={"ID":"25afb96c-6351-470d-8dfa-dd8a9145be40","Type":"ContainerStarted","Data":"c87ce8d555e9bc090049a675fd0023eb53d730a4c298de5166748ed307fa3802"} Oct 29 23:12:54.952522 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.952412 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:12:54.972319 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:54.972293 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm"] Oct 29 23:12:55.063949 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.063917 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:55.067069 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.067043 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:55.158516 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.158483 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:55.280396 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.280322 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:12:55.362191 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.362126 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:55.366731 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.366708 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:55.367892 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.367870 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-866787b6f9-24krt" Oct 29 23:12:55.515632 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.515547 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:55.515809 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.515643 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:55.515809 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.515678 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:55.515809 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:55.515704 5056 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 29 23:12:55.515809 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.515715 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:55.515809 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:55.515773 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls podName:63784b03-969b-4e2a-b799-e027ad1c0183 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.515755196 +0000 UTC m=+4.882502829 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-t6gcj" (UID: "63784b03-969b-4e2a-b799-e027ad1c0183") : secret "cluster-monitoring-operator-tls" not found Oct 29 23:12:55.516069 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:55.515827 5056 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 29 23:12:55.516069 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:55.515887 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls podName:b3b53bf9-0305-43ad-99b1-26f35c841020 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.515873878 +0000 UTC m=+4.882621504 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x975f" (UID: "b3b53bf9-0305-43ad-99b1-26f35c841020") : secret "insights-runtime-extractor-tls" not found Oct 29 23:12:55.516069 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:55.515932 5056 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 29 23:12:55.516069 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:55.515962 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm podName:b3b53bf9-0305-43ad-99b1-26f35c841020 nodeName:}" failed. No retries permitted until 2026-10-29 23:12:57.515951772 +0000 UTC m=+4.882699399 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x975f" (UID: "b3b53bf9-0305-43ad-99b1-26f35c841020") : configmap "kube-rbac-proxy" not found Oct 29 23:12:55.518634 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.518580 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c6fc8782-2d71-4cb3-ab4b-b1d14aad460f-metrics-certs\") pod \"network-metrics-daemon-2xktq\" (UID: \"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f\") " pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:55.566308 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.566078 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2xktq" Oct 29 23:12:55.573748 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.573681 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:55.769538 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.769355 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:55.854899 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.854872 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/2.log" Oct 29 23:12:55.855419 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.855350 5056 scope.go:122] "RemoveContainer" containerID="35ddcb9d9abd41d673ad3528ea7c3eaff6e432f5bd52475a5b11ddf923e5a122" Oct 29 23:12:55.855609 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:55.855586 5056 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2gzh7_openshift-console-operator(947dc777-7dd0-4244-913a-5dd068e6e6a3)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" podUID="947dc777-7dd0-4244-913a-5dd068e6e6a3" Oct 29 23:12:55.857981 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.857952 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" event={"ID":"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4","Type":"ContainerStarted","Data":"3362ec4ee05b50c7aa3ee96a95370e99de273e514f0d40dc2741efbaa020935c"} Oct 29 23:12:55.858078 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.858024 5056 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 29 23:12:55.981479 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:55.981445 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2xktq"] Oct 29 23:12:56.009732 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:56.009697 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6fc8782_2d71_4cb3_ab4b_b1d14aad460f.slice/crio-862a4ba02ca5d2ccb3f38eb0f23b137b8688a46cb61e8f26cfcb52f04cfa2dc2 WatchSource:0}: Error finding container 862a4ba02ca5d2ccb3f38eb0f23b137b8688a46cb61e8f26cfcb52f04cfa2dc2: Status 404 returned error can't find the container with id 862a4ba02ca5d2ccb3f38eb0f23b137b8688a46cb61e8f26cfcb52f04cfa2dc2 Oct 29 23:12:56.032563 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:56.032497 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:56.801844 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:56.801774 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:12:56.802008 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:56.801995 5056 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 29 23:12:56.865283 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:56.865226 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-b9jng" event={"ID":"b218e5d9-83c0-4976-90da-471e6b86d460","Type":"ContainerStarted","Data":"1c2e8689b5ae687b4a70aead0c745eb1d44fa6ca778dbdb78bbf554fa9a15345"} Oct 29 23:12:56.866717 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:56.866695 5056 generic.go:356] "Generic (PLEG): container finished" podID="ad5dcaa4-c226-4639-92fe-ace864d049ac" containerID="ba6a74650dec9da409edd2d622173de72d3a67137ca3f0d1417d535ce7437ab7" exitCode=0 Oct 29 23:12:56.866793 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:56.866769 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-m72dz" event={"ID":"ad5dcaa4-c226-4639-92fe-ace864d049ac","Type":"ContainerDied","Data":"ba6a74650dec9da409edd2d622173de72d3a67137ca3f0d1417d535ce7437ab7"} Oct 29 23:12:56.867077 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:56.867063 5056 scope.go:122] "RemoveContainer" containerID="ba6a74650dec9da409edd2d622173de72d3a67137ca3f0d1417d535ce7437ab7" Oct 29 23:12:56.868002 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:56.867979 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2xktq" event={"ID":"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f","Type":"ContainerStarted","Data":"862a4ba02ca5d2ccb3f38eb0f23b137b8688a46cb61e8f26cfcb52f04cfa2dc2"} Oct 29 23:12:56.868281 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:56.868251 5056 scope.go:122] "RemoveContainer" containerID="35ddcb9d9abd41d673ad3528ea7c3eaff6e432f5bd52475a5b11ddf923e5a122" Oct 29 23:12:56.868444 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:56.868426 5056 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2gzh7_openshift-console-operator(947dc777-7dd0-4244-913a-5dd068e6e6a3)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" podUID="947dc777-7dd0-4244-913a-5dd068e6e6a3" Oct 29 23:12:57.179693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:57.179667 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:12:57.227407 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:57.227380 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:57.438526 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:57.438466 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:57.537295 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:57.537225 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:57.537435 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:57.537336 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:57.537435 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:57.537382 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:12:57.537587 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:57.537549 5056 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 29 23:12:57.537670 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:57.537605 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls podName:b3b53bf9-0305-43ad-99b1-26f35c841020 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:01.537584787 +0000 UTC m=+8.904332414 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x975f" (UID: "b3b53bf9-0305-43ad-99b1-26f35c841020") : secret "insights-runtime-extractor-tls" not found Oct 29 23:12:57.537912 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:57.537821 5056 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 29 23:12:57.537912 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:12:57.537887 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm podName:b3b53bf9-0305-43ad-99b1-26f35c841020 nodeName:}" failed. No retries permitted until 2026-10-29 23:13:01.53787209 +0000 UTC m=+8.904619702 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x975f" (UID: "b3b53bf9-0305-43ad-99b1-26f35c841020") : configmap "kube-rbac-proxy" not found Oct 29 23:12:57.540002 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:57.539983 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/63784b03-969b-4e2a-b799-e027ad1c0183-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-t6gcj\" (UID: \"63784b03-969b-4e2a-b799-e027ad1c0183\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:57.625091 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:57.625049 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:57.660424 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:57.660393 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" Oct 29 23:12:57.882284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:57.881810 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:57.884440 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:57.884411 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-m72dz" event={"ID":"ad5dcaa4-c226-4639-92fe-ace864d049ac","Type":"ContainerStarted","Data":"a3925dc76a949847903e350a5ec80751992b92232d58faeb092832aaf9e91c74"} Oct 29 23:12:57.886700 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:57.886668 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" event={"ID":"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4","Type":"ContainerStarted","Data":"a1dbcdafb2199e62f68c781871b58ef308a1d23094ff5d00367cef31dc8cc521"} Oct 29 23:12:57.886779 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:57.886718 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-jfvvm" event={"ID":"8afad6d3-d1a6-4cf1-83e7-2f7e69bc20d4","Type":"ContainerStarted","Data":"4aef8a3226d91dbd75a8bea3d9ec28688cbd333e397e40ae6da93384b6af6ff9"} Oct 29 23:12:57.890885 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:57.890859 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gnwhl" event={"ID":"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93","Type":"ContainerStarted","Data":"13651537a4652c9b9840ec407fa8e8cc69c873a354326319ee8fed86ae989f64"} Oct 29 23:12:57.890954 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:57.890914 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gnwhl" event={"ID":"8388282a-8b2c-4f5c-ab3d-ec58ea20cc93","Type":"ContainerStarted","Data":"d62d651f2d0891b68eeff094495d837c9596227550a11f75874d76d2d3aaea6c"} Oct 29 23:12:58.076778 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:58.076720 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:58.115359 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:58.115339 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj"] Oct 29 23:12:58.117845 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:12:58.117814 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63784b03_969b_4e2a_b799_e027ad1c0183.slice/crio-1c6dd5b974b35727d84c6e0eb1ef8cec003af81296e8beb4c1e03a39e3c90fed WatchSource:0}: Error finding container 1c6dd5b974b35727d84c6e0eb1ef8cec003af81296e8beb4c1e03a39e3c90fed: Status 404 returned error can't find the container with id 1c6dd5b974b35727d84c6e0eb1ef8cec003af81296e8beb4c1e03a39e3c90fed Oct 29 23:12:58.405490 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:58.405455 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:58.519321 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:58.519282 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:12:58.897927 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:58.897892 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" event={"ID":"63784b03-969b-4e2a-b799-e027ad1c0183","Type":"ContainerStarted","Data":"1c6dd5b974b35727d84c6e0eb1ef8cec003af81296e8beb4c1e03a39e3c90fed"} Oct 29 23:12:58.900018 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:58.899889 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2xktq" event={"ID":"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f","Type":"ContainerStarted","Data":"c50d53181b44b499f7aa919adf942c7562ca867ed71a07fa368d385b2cfe8416"} Oct 29 23:12:58.900018 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:58.899946 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2xktq" event={"ID":"c6fc8782-2d71-4cb3-ab4b-b1d14aad460f","Type":"ContainerStarted","Data":"1f93bd13eb130401bce4e3259af9c259b5df817b33442e6dff9607288a64ef97"} Oct 29 23:12:58.900400 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:58.900380 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gnwhl" Oct 29 23:12:59.033404 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:59.033367 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:59.266237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:59.266153 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:59.431448 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:59.431411 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:59.594372 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:59.594275 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:12:59.905055 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:59.905023 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-t6gcj" event={"ID":"63784b03-969b-4e2a-b799-e027ad1c0183","Type":"ContainerStarted","Data":"ccdb17b9985084562ecc5f33381f157291488471cdafecf6d123f8ae8558fb3e"} Oct 29 23:12:59.972014 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:12:59.971980 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:00.279177 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:00.279103 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:00.462453 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:00.460768 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:00.671279 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:00.671230 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:00.831704 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:00.831669 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:01.514256 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:01.514222 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:01.572465 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:01.572434 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:13:01.572618 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:01.572520 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:13:01.573055 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:01.573027 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3b53bf9-0305-43ad-99b1-26f35c841020-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:13:01.574775 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:01.574757 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3b53bf9-0305-43ad-99b1-26f35c841020-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x975f\" (UID: \"b3b53bf9-0305-43ad-99b1-26f35c841020\") " pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:13:01.577760 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:01.577733 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x975f" Oct 29 23:13:01.875680 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:01.875546 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x975f"] Oct 29 23:13:01.878354 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:13:01.878322 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3b53bf9_0305_43ad_99b1_26f35c841020.slice/crio-88515114f3545852b776139558f484bae6bfbdd1170f6230be42413be20fcb79 WatchSource:0}: Error finding container 88515114f3545852b776139558f484bae6bfbdd1170f6230be42413be20fcb79: Status 404 returned error can't find the container with id 88515114f3545852b776139558f484bae6bfbdd1170f6230be42413be20fcb79 Oct 29 23:13:01.913896 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:01.913863 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x975f" event={"ID":"b3b53bf9-0305-43ad-99b1-26f35c841020","Type":"ContainerStarted","Data":"88515114f3545852b776139558f484bae6bfbdd1170f6230be42413be20fcb79"} Oct 29 23:13:02.918838 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:02.918800 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x975f" event={"ID":"b3b53bf9-0305-43ad-99b1-26f35c841020","Type":"ContainerStarted","Data":"7c8130eec60b3da39d70452a0e9c90013300a049c02d6b7c67019383999b010d"} Oct 29 23:13:02.919292 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:02.918863 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x975f" event={"ID":"b3b53bf9-0305-43ad-99b1-26f35c841020","Type":"ContainerStarted","Data":"4edd748ebaf43771f109a44c7a41694a71120d7e2d674ec4ad2fa5eb20d40507"} Oct 29 23:13:03.924943 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:03.924907 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x975f" event={"ID":"b3b53bf9-0305-43ad-99b1-26f35c841020","Type":"ContainerStarted","Data":"8b682df122b8d5afdaeaff1d101bf5e31cf722ca6e8e78655689ef6e4c8ac48e"} Oct 29 23:13:04.233458 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:04.233415 5056 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:13:04.233827 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:04.233812 5056 scope.go:122] "RemoveContainer" containerID="35ddcb9d9abd41d673ad3528ea7c3eaff6e432f5bd52475a5b11ddf923e5a122" Oct 29 23:13:04.930982 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:04.930948 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:13:04.931419 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:04.931394 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/2.log" Oct 29 23:13:04.931462 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:04.931426 5056 generic.go:356] "Generic (PLEG): container finished" podID="947dc777-7dd0-4244-913a-5dd068e6e6a3" containerID="bbb47370a197d226c079092e7e7d8043b060695918991c18d9abdae0167e65a1" exitCode=255 Oct 29 23:13:04.931525 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:04.931503 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" event={"ID":"947dc777-7dd0-4244-913a-5dd068e6e6a3","Type":"ContainerDied","Data":"bbb47370a197d226c079092e7e7d8043b060695918991c18d9abdae0167e65a1"} Oct 29 23:13:04.931573 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:04.931563 5056 scope.go:122] "RemoveContainer" containerID="35ddcb9d9abd41d673ad3528ea7c3eaff6e432f5bd52475a5b11ddf923e5a122" Oct 29 23:13:04.931966 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:04.931943 5056 scope.go:122] "RemoveContainer" containerID="bbb47370a197d226c079092e7e7d8043b060695918991c18d9abdae0167e65a1" Oct 29 23:13:04.932150 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:13:04.932132 5056 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2gzh7_openshift-console-operator(947dc777-7dd0-4244-913a-5dd068e6e6a3)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" podUID="947dc777-7dd0-4244-913a-5dd068e6e6a3" Oct 29 23:13:04.953358 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:04.953335 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:13:05.393574 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:05.393549 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-25m4p" Oct 29 23:13:05.737827 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:05.737719 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:05.893845 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:05.893790 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:05.937391 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:05.937363 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:13:05.937768 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:05.937683 5056 scope.go:122] "RemoveContainer" containerID="bbb47370a197d226c079092e7e7d8043b060695918991c18d9abdae0167e65a1" Oct 29 23:13:05.937889 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:13:05.937869 5056 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2gzh7_openshift-console-operator(947dc777-7dd0-4244-913a-5dd068e6e6a3)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" podUID="947dc777-7dd0-4244-913a-5dd068e6e6a3" Oct 29 23:13:08.907122 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:08.907085 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gnwhl" Oct 29 23:13:08.917999 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:08.917965 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-9jt7p"] Oct 29 23:13:08.923150 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:08.923123 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-9jt7p" Oct 29 23:13:08.941294 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:08.940802 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5679dbd569-548pj"] Oct 29 23:13:08.942021 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:08.941799 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-mhdnw-4d4k4"] Oct 29 23:13:08.946555 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:08.946533 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" Oct 29 23:13:08.952092 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:08.952030 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-l9r7k\"" Oct 29 23:13:08.953252 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:08.952455 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Oct 29 23:13:08.970420 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:08.970389 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-9jt7p"] Oct 29 23:13:08.977342 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:08.970818 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-mhdnw-4d4k4"] Oct 29 23:13:09.029797 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.029768 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5768db7d69-j4qs9"] Oct 29 23:13:09.033036 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.033013 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.043156 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.043133 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e91b184c-b290-4c7e-a763-52c80c34a17d-registry-tls\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.043306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.043166 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e91b184c-b290-4c7e-a763-52c80c34a17d-bound-sa-token\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.043306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.043194 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e91b184c-b290-4c7e-a763-52c80c34a17d-image-registry-private-configuration\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.043306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.043291 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e91b184c-b290-4c7e-a763-52c80c34a17d-ca-trust-extracted\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.043425 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.043335 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e91b184c-b290-4c7e-a763-52c80c34a17d-registry-certificates\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.043425 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.043381 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e91b184c-b290-4c7e-a763-52c80c34a17d-trusted-ca\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.043425 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.043396 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc78n\" (UniqueName: \"kubernetes.io/projected/e91b184c-b290-4c7e-a763-52c80c34a17d-kube-api-access-fc78n\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.043537 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.043477 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e91b184c-b290-4c7e-a763-52c80c34a17d-installation-pull-secrets\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.043537 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.043499 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7bc8c39-796f-40df-bca0-eddc3eab3116-service-ca-bundle\") pod \"periodic-gathering-mhdnw-4d4k4\" (UID: \"a7bc8c39-796f-40df-bca0-eddc3eab3116\") " pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" Oct 29 23:13:09.043537 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.043517 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2af4443b-bbec-448d-9a41-07fcc122059b-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-9jt7p\" (UID: \"2af4443b-bbec-448d-9a41-07fcc122059b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-9jt7p" Oct 29 23:13:09.043645 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.043537 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7bc8c39-796f-40df-bca0-eddc3eab3116-trusted-ca-bundle\") pod \"periodic-gathering-mhdnw-4d4k4\" (UID: \"a7bc8c39-796f-40df-bca0-eddc3eab3116\") " pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" Oct 29 23:13:09.043645 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.043595 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6v5s\" (UniqueName: \"kubernetes.io/projected/a7bc8c39-796f-40df-bca0-eddc3eab3116-kube-api-access-c6v5s\") pod \"periodic-gathering-mhdnw-4d4k4\" (UID: \"a7bc8c39-796f-40df-bca0-eddc3eab3116\") " pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" Oct 29 23:13:09.043645 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.043636 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a7bc8c39-796f-40df-bca0-eddc3eab3116-archives-path\") pod \"periodic-gathering-mhdnw-4d4k4\" (UID: \"a7bc8c39-796f-40df-bca0-eddc3eab3116\") " pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" Oct 29 23:13:09.052379 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.052336 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5768db7d69-j4qs9"] Oct 29 23:13:09.083714 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.083688 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:09.144221 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.144193 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2af4443b-bbec-448d-9a41-07fcc122059b-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-9jt7p\" (UID: \"2af4443b-bbec-448d-9a41-07fcc122059b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-9jt7p" Oct 29 23:13:09.144401 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.144227 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7bc8c39-796f-40df-bca0-eddc3eab3116-trusted-ca-bundle\") pod \"periodic-gathering-mhdnw-4d4k4\" (UID: \"a7bc8c39-796f-40df-bca0-eddc3eab3116\") " pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" Oct 29 23:13:09.144401 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.144248 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6v5s\" (UniqueName: \"kubernetes.io/projected/a7bc8c39-796f-40df-bca0-eddc3eab3116-kube-api-access-c6v5s\") pod \"periodic-gathering-mhdnw-4d4k4\" (UID: \"a7bc8c39-796f-40df-bca0-eddc3eab3116\") " pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" Oct 29 23:13:09.144401 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.144283 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a7bc8c39-796f-40df-bca0-eddc3eab3116-archives-path\") pod \"periodic-gathering-mhdnw-4d4k4\" (UID: \"a7bc8c39-796f-40df-bca0-eddc3eab3116\") " pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" Oct 29 23:13:09.144401 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.144303 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e91b184c-b290-4c7e-a763-52c80c34a17d-registry-tls\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.144401 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.144330 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e91b184c-b290-4c7e-a763-52c80c34a17d-bound-sa-token\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.144401 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.144358 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e91b184c-b290-4c7e-a763-52c80c34a17d-image-registry-private-configuration\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.144401 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.144381 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e91b184c-b290-4c7e-a763-52c80c34a17d-ca-trust-extracted\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.144762 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.144407 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e91b184c-b290-4c7e-a763-52c80c34a17d-registry-certificates\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.144762 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.144474 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e91b184c-b290-4c7e-a763-52c80c34a17d-trusted-ca\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.144762 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.144498 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc78n\" (UniqueName: \"kubernetes.io/projected/e91b184c-b290-4c7e-a763-52c80c34a17d-kube-api-access-fc78n\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.144762 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.144552 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e91b184c-b290-4c7e-a763-52c80c34a17d-installation-pull-secrets\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.144762 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.144580 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7bc8c39-796f-40df-bca0-eddc3eab3116-service-ca-bundle\") pod \"periodic-gathering-mhdnw-4d4k4\" (UID: \"a7bc8c39-796f-40df-bca0-eddc3eab3116\") " pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" Oct 29 23:13:09.144762 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.144712 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a7bc8c39-796f-40df-bca0-eddc3eab3116-archives-path\") pod \"periodic-gathering-mhdnw-4d4k4\" (UID: \"a7bc8c39-796f-40df-bca0-eddc3eab3116\") " pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" Oct 29 23:13:09.145291 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.145220 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7bc8c39-796f-40df-bca0-eddc3eab3116-service-ca-bundle\") pod \"periodic-gathering-mhdnw-4d4k4\" (UID: \"a7bc8c39-796f-40df-bca0-eddc3eab3116\") " pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" Oct 29 23:13:09.145450 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.145421 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e91b184c-b290-4c7e-a763-52c80c34a17d-ca-trust-extracted\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.145515 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.145435 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7bc8c39-796f-40df-bca0-eddc3eab3116-trusted-ca-bundle\") pod \"periodic-gathering-mhdnw-4d4k4\" (UID: \"a7bc8c39-796f-40df-bca0-eddc3eab3116\") " pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" Oct 29 23:13:09.145622 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.145524 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e91b184c-b290-4c7e-a763-52c80c34a17d-registry-certificates\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.145864 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.145838 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e91b184c-b290-4c7e-a763-52c80c34a17d-trusted-ca\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.147669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.147510 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e91b184c-b290-4c7e-a763-52c80c34a17d-installation-pull-secrets\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.147669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.147625 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e91b184c-b290-4c7e-a763-52c80c34a17d-image-registry-private-configuration\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.147789 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.147674 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/2af4443b-bbec-448d-9a41-07fcc122059b-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-9jt7p\" (UID: \"2af4443b-bbec-448d-9a41-07fcc122059b\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-9jt7p" Oct 29 23:13:09.148057 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.148034 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e91b184c-b290-4c7e-a763-52c80c34a17d-registry-tls\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.184654 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.184582 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e91b184c-b290-4c7e-a763-52c80c34a17d-bound-sa-token\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.184773 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.184748 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6v5s\" (UniqueName: \"kubernetes.io/projected/a7bc8c39-796f-40df-bca0-eddc3eab3116-kube-api-access-c6v5s\") pod \"periodic-gathering-mhdnw-4d4k4\" (UID: \"a7bc8c39-796f-40df-bca0-eddc3eab3116\") " pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" Oct 29 23:13:09.184773 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.184765 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc78n\" (UniqueName: \"kubernetes.io/projected/e91b184c-b290-4c7e-a763-52c80c34a17d-kube-api-access-fc78n\") pod \"image-registry-5768db7d69-j4qs9\" (UID: \"e91b184c-b290-4c7e-a763-52c80c34a17d\") " pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.266603 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.266350 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-9jt7p" Oct 29 23:13:09.281883 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.281858 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" Oct 29 23:13:09.351117 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.351084 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.385290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.385219 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:09.584177 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.584102 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:09.589280 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.589241 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-9jt7p"] Oct 29 23:13:09.591163 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:13:09.591135 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2af4443b_bbec_448d_9a41_07fcc122059b.slice/crio-e59d6a4ccb4e27246a1fe049c548899ce50fff4ef5ad531808a0e93efd3f889d WatchSource:0}: Error finding container e59d6a4ccb4e27246a1fe049c548899ce50fff4ef5ad531808a0e93efd3f889d: Status 404 returned error can't find the container with id e59d6a4ccb4e27246a1fe049c548899ce50fff4ef5ad531808a0e93efd3f889d Oct 29 23:13:09.609224 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.609206 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-mhdnw-4d4k4"] Oct 29 23:13:09.610741 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:13:09.610712 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7bc8c39_796f_40df_bca0_eddc3eab3116.slice/crio-1494453991a43a5b6f5c4117a0b4c541ed0ddbfd0848d19f6036ed8d35f82aa7 WatchSource:0}: Error finding container 1494453991a43a5b6f5c4117a0b4c541ed0ddbfd0848d19f6036ed8d35f82aa7: Status 404 returned error can't find the container with id 1494453991a43a5b6f5c4117a0b4c541ed0ddbfd0848d19f6036ed8d35f82aa7 Oct 29 23:13:09.681768 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.681745 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5768db7d69-j4qs9"] Oct 29 23:13:09.684238 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:13:09.684204 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode91b184c_b290_4c7e_a763_52c80c34a17d.slice/crio-e786cc01bd8cc89568cf87b471fdda883c0a288851e2dbc7cc65d8ce8f749998 WatchSource:0}: Error finding container e786cc01bd8cc89568cf87b471fdda883c0a288851e2dbc7cc65d8ce8f749998: Status 404 returned error can't find the container with id e786cc01bd8cc89568cf87b471fdda883c0a288851e2dbc7cc65d8ce8f749998 Oct 29 23:13:09.820467 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.820427 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:09.966846 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.966813 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-9jt7p" event={"ID":"2af4443b-bbec-448d-9a41-07fcc122059b","Type":"ContainerStarted","Data":"e59d6a4ccb4e27246a1fe049c548899ce50fff4ef5ad531808a0e93efd3f889d"} Oct 29 23:13:09.968505 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.968473 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" event={"ID":"e91b184c-b290-4c7e-a763-52c80c34a17d","Type":"ContainerStarted","Data":"abc971d26655d4498dad3b2bf368140d482f4cd06a978a447ec262bbd66d444f"} Oct 29 23:13:09.968706 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.968683 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:09.968811 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.968715 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" event={"ID":"e91b184c-b290-4c7e-a763-52c80c34a17d","Type":"ContainerStarted","Data":"e786cc01bd8cc89568cf87b471fdda883c0a288851e2dbc7cc65d8ce8f749998"} Oct 29 23:13:09.970241 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.970214 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" event={"ID":"a7bc8c39-796f-40df-bca0-eddc3eab3116","Type":"ContainerStarted","Data":"bb99e9b0eda2fbe2dbbf94cd6a9639ec8286829091abacf6833515001f4fc3af"} Oct 29 23:13:09.970359 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:09.970280 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" event={"ID":"a7bc8c39-796f-40df-bca0-eddc3eab3116","Type":"ContainerStarted","Data":"1494453991a43a5b6f5c4117a0b4c541ed0ddbfd0848d19f6036ed8d35f82aa7"} Oct 29 23:13:10.016698 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:10.016646 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" podStartSLOduration=2.016633012 podStartE2EDuration="2.016633012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:10.016327751 +0000 UTC m=+17.383075382" watchObservedRunningTime="2026-10-29 23:13:10.016633012 +0000 UTC m=+17.383380637" Oct 29 23:13:10.068143 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:10.061290 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" podStartSLOduration=2.061253823 podStartE2EDuration="2.061253823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:10.059932739 +0000 UTC m=+17.426680369" watchObservedRunningTime="2026-10-29 23:13:10.061253823 +0000 UTC m=+17.428001453" Oct 29 23:13:10.591781 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:10.591752 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gnwhl_8388282a-8b2c-4f5c-ab3d-ec58ea20cc93/dns/0.log" Oct 29 23:13:10.795466 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:10.795439 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gnwhl_8388282a-8b2c-4f5c-ab3d-ec58ea20cc93/kube-rbac-proxy/0.log" Oct 29 23:13:10.976209 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:10.976169 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-9jt7p" event={"ID":"2af4443b-bbec-448d-9a41-07fcc122059b","Type":"ContainerStarted","Data":"3c6457abf3bac7694cd8f13ff95b015327c89c0387d520f84f7f0112a96c346a"} Oct 29 23:13:10.976646 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:10.976242 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-9jt7p" Oct 29 23:13:10.981234 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:10.981211 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-9jt7p" Oct 29 23:13:11.021307 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.021236 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-9jt7p" podStartSLOduration=2.03922413 podStartE2EDuration="3.021223694s" totalImagesPullingTime="981.999564ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:11.020128807 +0000 UTC m=+18.386876439" watchObservedRunningTime="2026-10-29 23:13:11.021223694 +0000 UTC m=+18.387971324" Oct 29 23:13:11.189774 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.189745 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zr8dl_1211641d-1d2c-4de0-b993-5f1e84edc09f/dns-node-resolver/0.log" Oct 29 23:13:11.589216 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.589186 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5679dbd569-548pj_6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c/registry/0.log" Oct 29 23:13:11.753011 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.752977 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:11.790430 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.790403 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5768db7d69-j4qs9_e91b184c-b290-4c7e-a763-52c80c34a17d/registry/0.log" Oct 29 23:13:11.807965 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.807933 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt"] Oct 29 23:13:11.811404 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.811384 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" Oct 29 23:13:11.843770 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.843661 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Oct 29 23:13:11.843770 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.843698 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Oct 29 23:13:11.843770 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.843719 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 29 23:13:11.844710 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.844690 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt"] Oct 29 23:13:11.846464 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.846446 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-mz66n\"" Oct 29 23:13:11.868743 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.868719 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9cb5fa50-deea-4044-a64e-fa6ea83bb1c4-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-6p9zt\" (UID: \"9cb5fa50-deea-4044-a64e-fa6ea83bb1c4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" Oct 29 23:13:11.868850 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.868749 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbdrq\" (UniqueName: \"kubernetes.io/projected/9cb5fa50-deea-4044-a64e-fa6ea83bb1c4-kube-api-access-zbdrq\") pod \"prometheus-operator-9cd89b74c-6p9zt\" (UID: \"9cb5fa50-deea-4044-a64e-fa6ea83bb1c4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" Oct 29 23:13:11.868850 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.868776 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9cb5fa50-deea-4044-a64e-fa6ea83bb1c4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-6p9zt\" (UID: \"9cb5fa50-deea-4044-a64e-fa6ea83bb1c4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" Oct 29 23:13:11.868940 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.868890 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cb5fa50-deea-4044-a64e-fa6ea83bb1c4-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-6p9zt\" (UID: \"9cb5fa50-deea-4044-a64e-fa6ea83bb1c4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" Oct 29 23:13:11.914164 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.914129 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:11.969447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.969416 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9cb5fa50-deea-4044-a64e-fa6ea83bb1c4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-6p9zt\" (UID: \"9cb5fa50-deea-4044-a64e-fa6ea83bb1c4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" Oct 29 23:13:11.969610 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.969459 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cb5fa50-deea-4044-a64e-fa6ea83bb1c4-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-6p9zt\" (UID: \"9cb5fa50-deea-4044-a64e-fa6ea83bb1c4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" Oct 29 23:13:11.969610 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.969545 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9cb5fa50-deea-4044-a64e-fa6ea83bb1c4-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-6p9zt\" (UID: \"9cb5fa50-deea-4044-a64e-fa6ea83bb1c4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" Oct 29 23:13:11.969610 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.969573 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbdrq\" (UniqueName: \"kubernetes.io/projected/9cb5fa50-deea-4044-a64e-fa6ea83bb1c4-kube-api-access-zbdrq\") pod \"prometheus-operator-9cd89b74c-6p9zt\" (UID: \"9cb5fa50-deea-4044-a64e-fa6ea83bb1c4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" Oct 29 23:13:11.970222 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.970193 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9cb5fa50-deea-4044-a64e-fa6ea83bb1c4-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-6p9zt\" (UID: \"9cb5fa50-deea-4044-a64e-fa6ea83bb1c4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" Oct 29 23:13:11.972002 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.971982 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9cb5fa50-deea-4044-a64e-fa6ea83bb1c4-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-6p9zt\" (UID: \"9cb5fa50-deea-4044-a64e-fa6ea83bb1c4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" Oct 29 23:13:11.972085 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.972062 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/9cb5fa50-deea-4044-a64e-fa6ea83bb1c4-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-6p9zt\" (UID: \"9cb5fa50-deea-4044-a64e-fa6ea83bb1c4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" Oct 29 23:13:11.988917 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.988890 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7cbdc4b876-9xjlm_25afb96c-6351-470d-8dfa-dd8a9145be40/registry/0.log" Oct 29 23:13:11.994668 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:11.994639 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbdrq\" (UniqueName: \"kubernetes.io/projected/9cb5fa50-deea-4044-a64e-fa6ea83bb1c4-kube-api-access-zbdrq\") pod \"prometheus-operator-9cd89b74c-6p9zt\" (UID: \"9cb5fa50-deea-4044-a64e-fa6ea83bb1c4\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" Oct 29 23:13:12.078438 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:12.078406 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:12.126463 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:12.126387 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" Oct 29 23:13:12.188886 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:12.188863 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jg249_c652cee1-24f4-460a-8e1a-669b0ae9c0b1/node-ca/0.log" Oct 29 23:13:12.296092 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:12.296055 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:12.424115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:12.424088 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt"] Oct 29 23:13:12.426351 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:13:12.426321 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb5fa50_deea_4044_a64e_fa6ea83bb1c4.slice/crio-abfb8d5da3fdb646dde0fed87b8296f8c0ba553390035828f560d05ae5659583 WatchSource:0}: Error finding container abfb8d5da3fdb646dde0fed87b8296f8c0ba553390035828f560d05ae5659583: Status 404 returned error can't find the container with id abfb8d5da3fdb646dde0fed87b8296f8c0ba553390035828f560d05ae5659583 Oct 29 23:13:12.474140 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:12.474112 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:12.659239 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:12.659202 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:12.789074 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:12.789004 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-866787b6f9-24krt_b6ad3759-e11c-476d-82ee-fffdf8fd6d47/router/0.log" Oct 29 23:13:12.907569 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:12.907531 5056 patch_prober.go:29] interesting pod/image-registry-7cbdc4b876-9xjlm container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Oct 29 23:13:12.907761 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:12.907592 5056 prober.go:122] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" podUID="25afb96c-6351-470d-8dfa-dd8a9145be40" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:13:12.988832 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:12.988759 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" event={"ID":"9cb5fa50-deea-4044-a64e-fa6ea83bb1c4","Type":"ContainerStarted","Data":"abfb8d5da3fdb646dde0fed87b8296f8c0ba553390035828f560d05ae5659583"} Oct 29 23:13:12.989808 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:12.989781 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t8vj4_fa19a031-f171-495b-a72d-f4f79b88fceb/serve-healthcheck-canary/0.log" Oct 29 23:13:13.502841 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:13.502806 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:13.669712 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:13.669677 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:13.884623 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:13.884533 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:13.994598 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:13.994560 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" event={"ID":"9cb5fa50-deea-4044-a64e-fa6ea83bb1c4","Type":"ContainerStarted","Data":"7f8d41211b7b1fac7dd45c3cc3a7a677b4b4c683e665006532e099e1d05bf105"} Oct 29 23:13:13.995018 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:13.994613 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" event={"ID":"9cb5fa50-deea-4044-a64e-fa6ea83bb1c4","Type":"ContainerStarted","Data":"10ceff678e24ee04c829a4e41a8828085faf789eac791945f4b81e738648f6c6"} Oct 29 23:13:14.038096 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:14.038045 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-6p9zt" podStartSLOduration=1.9349888339999999 podStartE2EDuration="3.038033423s" totalImagesPullingTime="1.103044589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:13:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:14.037921515 +0000 UTC m=+21.404669145" watchObservedRunningTime="2026-10-29 23:13:14.038033423 +0000 UTC m=+21.404781050" Oct 29 23:13:14.059594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:14.059564 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:14.232448 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:14.232417 5056 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:13:14.232833 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:14.232819 5056 scope.go:122] "RemoveContainer" containerID="bbb47370a197d226c079092e7e7d8043b060695918991c18d9abdae0167e65a1" Oct 29 23:13:14.233033 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:13:14.233014 5056 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-2gzh7_openshift-console-operator(947dc777-7dd0-4244-913a-5dd068e6e6a3)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" podUID="947dc777-7dd0-4244-913a-5dd068e6e6a3" Oct 29 23:13:14.243526 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:14.243501 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:14.465511 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:14.465479 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:16.221663 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.221605 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:16.465515 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.465485 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:16.588656 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.588579 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-blg76" Oct 29 23:13:16.610764 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.610737 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-m5k4b"] Oct 29 23:13:16.648426 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.648399 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.664174 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.664146 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:16.666699 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.666675 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 29 23:13:16.677877 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.677855 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 29 23:13:16.677974 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.677855 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 29 23:13:16.677974 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.677951 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8hh8s\"" Oct 29 23:13:16.702610 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.702583 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a57ebca-6da5-46dc-9315-5896905b574d-metrics-client-ca\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.702705 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.702620 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7a57ebca-6da5-46dc-9315-5896905b574d-node-exporter-tls\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.702705 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.702646 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a57ebca-6da5-46dc-9315-5896905b574d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.702777 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.702744 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7a57ebca-6da5-46dc-9315-5896905b574d-node-exporter-textfile\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.702777 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.702772 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7a57ebca-6da5-46dc-9315-5896905b574d-node-exporter-wtmp\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.702838 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.702806 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7a57ebca-6da5-46dc-9315-5896905b574d-node-exporter-accelerators-collector-config\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.702925 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.702903 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7a57ebca-6da5-46dc-9315-5896905b574d-sys\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.702960 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.702932 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8hkq\" (UniqueName: \"kubernetes.io/projected/7a57ebca-6da5-46dc-9315-5896905b574d-kube-api-access-v8hkq\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.702960 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.702954 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7a57ebca-6da5-46dc-9315-5896905b574d-root\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.804510 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.804478 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7a57ebca-6da5-46dc-9315-5896905b574d-node-exporter-accelerators-collector-config\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.804682 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.804535 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7a57ebca-6da5-46dc-9315-5896905b574d-sys\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.804682 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.804563 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8hkq\" (UniqueName: \"kubernetes.io/projected/7a57ebca-6da5-46dc-9315-5896905b574d-kube-api-access-v8hkq\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.804682 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.804596 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7a57ebca-6da5-46dc-9315-5896905b574d-root\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.804682 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.804642 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a57ebca-6da5-46dc-9315-5896905b574d-metrics-client-ca\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.804962 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.804677 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7a57ebca-6da5-46dc-9315-5896905b574d-sys\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.804962 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.804670 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7a57ebca-6da5-46dc-9315-5896905b574d-node-exporter-tls\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.804962 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.804705 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7a57ebca-6da5-46dc-9315-5896905b574d-root\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.804962 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.804716 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a57ebca-6da5-46dc-9315-5896905b574d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.804962 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.804799 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7a57ebca-6da5-46dc-9315-5896905b574d-node-exporter-textfile\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.804962 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.804846 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7a57ebca-6da5-46dc-9315-5896905b574d-node-exporter-wtmp\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.805293 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.805003 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7a57ebca-6da5-46dc-9315-5896905b574d-node-exporter-wtmp\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.805293 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.805147 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7a57ebca-6da5-46dc-9315-5896905b574d-node-exporter-textfile\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.805293 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.805166 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7a57ebca-6da5-46dc-9315-5896905b574d-node-exporter-accelerators-collector-config\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.805457 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.805312 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a57ebca-6da5-46dc-9315-5896905b574d-metrics-client-ca\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.807106 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.807086 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7a57ebca-6da5-46dc-9315-5896905b574d-node-exporter-tls\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.807219 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.807138 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a57ebca-6da5-46dc-9315-5896905b574d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.829594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.829568 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8hkq\" (UniqueName: \"kubernetes.io/projected/7a57ebca-6da5-46dc-9315-5896905b574d-kube-api-access-v8hkq\") pod \"node-exporter-m5k4b\" (UID: \"7a57ebca-6da5-46dc-9315-5896905b574d\") " pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:16.848462 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:16.848377 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:17.001793 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:17.001764 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:17.016338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:17.016309 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m5k4b" Oct 29 23:13:17.051107 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:13:17.051077 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a57ebca_6da5_46dc_9315_5896905b574d.slice/crio-7d5a38217673125d94564dedd92fdc3f228795baab6ce912a26152d7769562b1 WatchSource:0}: Error finding container 7d5a38217673125d94564dedd92fdc3f228795baab6ce912a26152d7769562b1: Status 404 returned error can't find the container with id 7d5a38217673125d94564dedd92fdc3f228795baab6ce912a26152d7769562b1 Oct 29 23:13:17.156512 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:17.156480 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:17.185037 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:17.185011 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:13:17.600588 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:17.600483 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:17.760195 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:17.760161 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:17.946892 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:17.946863 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:18.012523 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:18.012490 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m5k4b" event={"ID":"7a57ebca-6da5-46dc-9315-5896905b574d","Type":"ContainerStarted","Data":"7d5a38217673125d94564dedd92fdc3f228795baab6ce912a26152d7769562b1"} Oct 29 23:13:18.389370 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:18.389319 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:18.568000 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:18.567973 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:18.790362 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:18.790323 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:18.947620 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:18.947592 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:13:18.990874 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:18.990841 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:19.017984 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:19.017953 5056 generic.go:356] "Generic (PLEG): container finished" podID="7a57ebca-6da5-46dc-9315-5896905b574d" containerID="30f66654207d4415d9381c1c31ef2d2703e3bf6020dd317400e14c767c9ebaa9" exitCode=0 Oct 29 23:13:19.018156 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:19.018029 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m5k4b" event={"ID":"7a57ebca-6da5-46dc-9315-5896905b574d","Type":"ContainerDied","Data":"30f66654207d4415d9381c1c31ef2d2703e3bf6020dd317400e14c767c9ebaa9"} Oct 29 23:13:19.802029 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:19.801972 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:20.024667 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:20.024634 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m5k4b" event={"ID":"7a57ebca-6da5-46dc-9315-5896905b574d","Type":"ContainerStarted","Data":"bc23a1356b5ff1c0df67d39bbede27b6c1902b4e323cb882944cf85e1630cd0a"} Oct 29 23:13:20.024835 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:20.024692 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m5k4b" event={"ID":"7a57ebca-6da5-46dc-9315-5896905b574d","Type":"ContainerStarted","Data":"97ea63a8b1fbe4db99b4183ed0b7a3ab35d02c9669ce015484749c299c3324ef"} Oct 29 23:13:20.032904 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:20.032880 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:20.081394 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:20.081291 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-m5k4b" podStartSLOduration=2.6854707749999998 podStartE2EDuration="4.08125421s" totalImagesPullingTime="1.314755263s" totalInitContainerRuntime="81.028172ms" isStatefulPod=true podCreationTimestamp="2026-10-29 23:13:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:13:20.08098667 +0000 UTC m=+27.447734301" watchObservedRunningTime="2026-10-29 23:13:20.08125421 +0000 UTC m=+27.448001890" Oct 29 23:13:20.206986 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:20.206948 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:20.366734 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:20.366652 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:21.039182 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:21.039144 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:21.221542 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:21.221506 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:21.412573 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:21.412537 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:21.611024 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:21.610990 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:22.414160 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:13:22.414123 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:13:23.474664 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:23.474634 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:23.761122 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:23.761039 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:23.919727 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:23.919690 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:25.446501 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:25.446466 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:25.724634 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:25.724549 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:27.426967 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:27.426935 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:27.805527 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:27.805442 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:28.522363 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:28.522333 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dn96d" Oct 29 23:13:28.770954 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:28.770917 5056 scope.go:122] "RemoveContainer" containerID="bbb47370a197d226c079092e7e7d8043b060695918991c18d9abdae0167e65a1" Oct 29 23:13:29.068062 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:29.067973 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:13:29.068245 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:29.068074 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" event={"ID":"947dc777-7dd0-4244-913a-5dd068e6e6a3","Type":"ContainerStarted","Data":"fc2e58b30417eaa12f811f7434910ebdcbfb3b28bcaf8ac08076f253200bab4f"} Oct 29 23:13:29.068546 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:29.068496 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:13:29.505346 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:29.505314 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-2gzh7" Oct 29 23:13:30.173432 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:30.173397 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:30.418581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:30.418550 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:30.865749 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:30.865688 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:30.982650 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:30.981744 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5768db7d69-j4qs9" Oct 29 23:13:31.110351 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:31.110246 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cbdc4b876-9xjlm"] Oct 29 23:13:31.492528 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:31.492492 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:31.783186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:31.783112 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:32.037755 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:32.037676 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:33.280997 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:33.280963 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:33.553854 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:33.553776 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:33.989183 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:33.989142 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5679dbd569-548pj" podUID="6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" containerName="registry" containerID="cri-o://c8660f151cf84005860e173c5498bfcdb28cb3171a1e49bfcdffd2d0d132964d" gracePeriod=30 Oct 29 23:13:34.251739 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.251681 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:13:34.383498 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.383425 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-image-registry-private-configuration\") pod \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " Oct 29 23:13:34.383928 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.383542 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls\") pod \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " Oct 29 23:13:34.383928 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.383595 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-ca-trust-extracted\") pod \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " Oct 29 23:13:34.383928 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.383687 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-kube-api-access-qsr2l\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-kube-api-access-qsr2l\") pod \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " Oct 29 23:13:34.383928 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.383743 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-trusted-ca\") pod \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " Oct 29 23:13:34.383928 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.383792 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-certificates\") pod \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " Oct 29 23:13:34.383928 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.383850 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-bound-sa-token\") pod \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " Oct 29 23:13:34.383928 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.383893 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-installation-pull-secrets\") pod \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\" (UID: \"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c\") " Oct 29 23:13:34.384340 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.384293 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-certificates" pod "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" (UID: "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:13:34.384733 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.384709 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-trusted-ca" pod "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" (UID: "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:13:34.385954 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.385932 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls" pod "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" (UID: "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:13:34.386486 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.386460 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-kube-api-access-qsr2l" pod "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" (UID: "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c"). InnerVolumeSpecName "kube-api-access-qsr2l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:13:34.386597 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.386466 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-installation-pull-secrets" pod "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" (UID: "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:13:34.386708 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.386687 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-bound-sa-token" pod "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" (UID: "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:13:34.386918 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.386897 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-image-registry-private-configuration" pod "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" (UID: "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:13:34.391037 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.391015 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-ca-trust-extracted" pod "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" (UID: "6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:13:34.485484 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.485443 5056 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-ca-trust-extracted\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:34.485484 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.485476 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsr2l\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-kube-api-access-qsr2l\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:34.485484 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.485487 5056 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-trusted-ca\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:34.485724 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.485496 5056 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-certificates\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:34.485724 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.485505 5056 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-bound-sa-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:34.485724 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.485514 5056 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-installation-pull-secrets\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:34.485724 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.485523 5056 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-image-registry-private-configuration\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:34.485724 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:34.485533 5056 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c-registry-tls\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:35.098152 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:35.098061 5056 generic.go:356] "Generic (PLEG): container finished" podID="6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" containerID="c8660f151cf84005860e173c5498bfcdb28cb3171a1e49bfcdffd2d0d132964d" exitCode=0 Oct 29 23:13:35.098152 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:35.098117 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5679dbd569-548pj" Oct 29 23:13:35.098152 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:35.098128 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5679dbd569-548pj" event={"ID":"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c","Type":"ContainerDied","Data":"c8660f151cf84005860e173c5498bfcdb28cb3171a1e49bfcdffd2d0d132964d"} Oct 29 23:13:35.098513 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:35.098166 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5679dbd569-548pj" event={"ID":"6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c","Type":"ContainerDied","Data":"cc1263d3065d9d848697d2002e7057f0b4a69cebaeacbb2b3d0b07766c2e7275"} Oct 29 23:13:35.098513 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:35.098185 5056 scope.go:122] "RemoveContainer" containerID="c8660f151cf84005860e173c5498bfcdb28cb3171a1e49bfcdffd2d0d132964d" Oct 29 23:13:35.106645 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:35.106628 5056 scope.go:122] "RemoveContainer" containerID="c8660f151cf84005860e173c5498bfcdb28cb3171a1e49bfcdffd2d0d132964d" Oct 29 23:13:35.106928 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:13:35.106900 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8660f151cf84005860e173c5498bfcdb28cb3171a1e49bfcdffd2d0d132964d\": container with ID starting with c8660f151cf84005860e173c5498bfcdb28cb3171a1e49bfcdffd2d0d132964d not found: ID does not exist" containerID="c8660f151cf84005860e173c5498bfcdb28cb3171a1e49bfcdffd2d0d132964d" Oct 29 23:13:35.106977 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:35.106939 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8660f151cf84005860e173c5498bfcdb28cb3171a1e49bfcdffd2d0d132964d"} err="failed to get container status \"c8660f151cf84005860e173c5498bfcdb28cb3171a1e49bfcdffd2d0d132964d\": rpc error: code = NotFound desc = could not find container \"c8660f151cf84005860e173c5498bfcdb28cb3171a1e49bfcdffd2d0d132964d\": container with ID starting with c8660f151cf84005860e173c5498bfcdb28cb3171a1e49bfcdffd2d0d132964d not found: ID does not exist" Oct 29 23:13:35.181717 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:35.181688 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5679dbd569-548pj"] Oct 29 23:13:35.194702 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:35.194677 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5679dbd569-548pj"] Oct 29 23:13:35.872177 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:35.872140 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:36.393893 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:36.393858 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:36.768116 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:36.768038 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" path="/var/lib/kubelet/pods/6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c/volumes" Oct 29 23:13:36.869530 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:36.869495 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:37.071712 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:37.071626 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:38.461463 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:38.461426 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:38.695873 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:38.695840 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:39.495946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:39.495913 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:39.816985 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:39.816897 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:41.811022 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:41.810982 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-hhksv-vlwlb"] Oct 29 23:13:41.811477 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:41.811214 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-hhksv-vlwlb" podUID="fce92e88-114c-49c7-b658-193d508126e9" containerName="insights-gathering" containerID="cri-o://a17b68680b921ac2171d902a12531b30bbec6f97957f90c0a88b4a9c5d68da08" gracePeriod=30 Oct 29 23:13:42.082421 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.082394 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:13:42.130108 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.130074 5056 generic.go:356] "Generic (PLEG): container finished" podID="fce92e88-114c-49c7-b658-193d508126e9" containerID="a17b68680b921ac2171d902a12531b30bbec6f97957f90c0a88b4a9c5d68da08" exitCode=2 Oct 29 23:13:42.130253 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.130129 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-hhksv-vlwlb" event={"ID":"fce92e88-114c-49c7-b658-193d508126e9","Type":"ContainerDied","Data":"a17b68680b921ac2171d902a12531b30bbec6f97957f90c0a88b4a9c5d68da08"} Oct 29 23:13:42.130253 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.130170 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-hhksv-vlwlb" event={"ID":"fce92e88-114c-49c7-b658-193d508126e9","Type":"ContainerDied","Data":"9c3adfcc48550f5cacb81cf8eb7f1bdd2f3ef037fb5cd65766debb6790819d43"} Oct 29 23:13:42.130253 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.130169 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-hhksv-vlwlb" Oct 29 23:13:42.130253 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.130188 5056 scope.go:122] "RemoveContainer" containerID="a17b68680b921ac2171d902a12531b30bbec6f97957f90c0a88b4a9c5d68da08" Oct 29 23:13:42.140633 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.140609 5056 scope.go:122] "RemoveContainer" containerID="a17b68680b921ac2171d902a12531b30bbec6f97957f90c0a88b4a9c5d68da08" Oct 29 23:13:42.140953 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:13:42.140929 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a17b68680b921ac2171d902a12531b30bbec6f97957f90c0a88b4a9c5d68da08\": container with ID starting with a17b68680b921ac2171d902a12531b30bbec6f97957f90c0a88b4a9c5d68da08 not found: ID does not exist" containerID="a17b68680b921ac2171d902a12531b30bbec6f97957f90c0a88b4a9c5d68da08" Oct 29 23:13:42.141060 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.140961 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a17b68680b921ac2171d902a12531b30bbec6f97957f90c0a88b4a9c5d68da08"} err="failed to get container status \"a17b68680b921ac2171d902a12531b30bbec6f97957f90c0a88b4a9c5d68da08\": rpc error: code = NotFound desc = could not find container \"a17b68680b921ac2171d902a12531b30bbec6f97957f90c0a88b4a9c5d68da08\": container with ID starting with a17b68680b921ac2171d902a12531b30bbec6f97957f90c0a88b4a9c5d68da08 not found: ID does not exist" Oct 29 23:13:42.153803 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.153779 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fce92e88-114c-49c7-b658-193d508126e9-kube-api-access-shjmc\" (UniqueName: \"kubernetes.io/projected/fce92e88-114c-49c7-b658-193d508126e9-kube-api-access-shjmc\") pod \"fce92e88-114c-49c7-b658-193d508126e9\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " Oct 29 23:13:42.153917 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.153854 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fce92e88-114c-49c7-b658-193d508126e9-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fce92e88-114c-49c7-b658-193d508126e9-archives-path\") pod \"fce92e88-114c-49c7-b658-193d508126e9\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " Oct 29 23:13:42.153965 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.153941 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-service-ca-bundle\") pod \"fce92e88-114c-49c7-b658-193d508126e9\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " Oct 29 23:13:42.154002 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.153982 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-trusted-ca-bundle\") pod \"fce92e88-114c-49c7-b658-193d508126e9\" (UID: \"fce92e88-114c-49c7-b658-193d508126e9\") " Oct 29 23:13:42.154328 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.154297 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fce92e88-114c-49c7-b658-193d508126e9-archives-path" pod "fce92e88-114c-49c7-b658-193d508126e9" (UID: "fce92e88-114c-49c7-b658-193d508126e9"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:13:42.154407 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.154317 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-service-ca-bundle" pod "fce92e88-114c-49c7-b658-193d508126e9" (UID: "fce92e88-114c-49c7-b658-193d508126e9"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:13:42.154407 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.154365 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-trusted-ca-bundle" pod "fce92e88-114c-49c7-b658-193d508126e9" (UID: "fce92e88-114c-49c7-b658-193d508126e9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:13:42.156136 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.156115 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fce92e88-114c-49c7-b658-193d508126e9-kube-api-access-shjmc" pod "fce92e88-114c-49c7-b658-193d508126e9" (UID: "fce92e88-114c-49c7-b658-193d508126e9"). InnerVolumeSpecName "kube-api-access-shjmc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:13:42.255438 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.255405 5056 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-service-ca-bundle\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:42.255438 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.255436 5056 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fce92e88-114c-49c7-b658-193d508126e9-trusted-ca-bundle\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:42.255438 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.255446 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shjmc\" (UniqueName: \"kubernetes.io/projected/fce92e88-114c-49c7-b658-193d508126e9-kube-api-access-shjmc\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:42.255714 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.255455 5056 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/fce92e88-114c-49c7-b658-193d508126e9-archives-path\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:42.444056 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.444022 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:42.476562 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.476535 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-hhksv-vlwlb"] Oct 29 23:13:42.489796 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.489772 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-hhksv-vlwlb"] Oct 29 23:13:42.626695 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.626661 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:42.768352 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:42.768277 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fce92e88-114c-49c7-b658-193d508126e9" path="/var/lib/kubelet/pods/fce92e88-114c-49c7-b658-193d508126e9/volumes" Oct 29 23:13:45.140500 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:45.140466 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:45.340781 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:45.340699 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:50.749937 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:50.749903 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:50.957708 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:50.957652 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:52.351236 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:52.351194 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:52.414102 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:13:52.414069 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:13:52.698634 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:52.698604 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:53.436988 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:53.436951 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:53.632081 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:53.632046 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:54.462197 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:54.462161 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:54.695462 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:54.695433 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:56.135431 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.130970 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" podUID="25afb96c-6351-470d-8dfa-dd8a9145be40" containerName="registry" containerID="cri-o://17bd8fa27348541385b4a3db0c1fccaadd53c1bff34b99bf555b8f9c3405c5a1" gracePeriod=30 Oct 29 23:13:56.328651 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.328616 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:56.381135 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.381111 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:13:56.481464 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.481436 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-trusted-ca\") pod \"25afb96c-6351-470d-8dfa-dd8a9145be40\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " Oct 29 23:13:56.481643 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.481477 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-kube-api-access-bzc9p\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-kube-api-access-bzc9p\") pod \"25afb96c-6351-470d-8dfa-dd8a9145be40\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " Oct 29 23:13:56.481643 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.481509 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-installation-pull-secrets\") pod \"25afb96c-6351-470d-8dfa-dd8a9145be40\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " Oct 29 23:13:56.481643 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.481539 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls\") pod \"25afb96c-6351-470d-8dfa-dd8a9145be40\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " Oct 29 23:13:56.481643 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.481586 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-certificates\") pod \"25afb96c-6351-470d-8dfa-dd8a9145be40\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " Oct 29 23:13:56.481643 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.481624 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-image-registry-private-configuration\") pod \"25afb96c-6351-470d-8dfa-dd8a9145be40\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " Oct 29 23:13:56.481909 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.481648 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-bound-sa-token\") pod \"25afb96c-6351-470d-8dfa-dd8a9145be40\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " Oct 29 23:13:56.481909 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.481689 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25afb96c-6351-470d-8dfa-dd8a9145be40-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/25afb96c-6351-470d-8dfa-dd8a9145be40-ca-trust-extracted\") pod \"25afb96c-6351-470d-8dfa-dd8a9145be40\" (UID: \"25afb96c-6351-470d-8dfa-dd8a9145be40\") " Oct 29 23:13:56.482075 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.482024 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-trusted-ca" pod "25afb96c-6351-470d-8dfa-dd8a9145be40" (UID: "25afb96c-6351-470d-8dfa-dd8a9145be40"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:13:56.482225 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.482185 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-certificates" pod "25afb96c-6351-470d-8dfa-dd8a9145be40" (UID: "25afb96c-6351-470d-8dfa-dd8a9145be40"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:13:56.484636 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.484603 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls" pod "25afb96c-6351-470d-8dfa-dd8a9145be40" (UID: "25afb96c-6351-470d-8dfa-dd8a9145be40"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:13:56.484739 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.484650 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-kube-api-access-bzc9p" pod "25afb96c-6351-470d-8dfa-dd8a9145be40" (UID: "25afb96c-6351-470d-8dfa-dd8a9145be40"). InnerVolumeSpecName "kube-api-access-bzc9p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:13:56.484739 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.484675 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-image-registry-private-configuration" pod "25afb96c-6351-470d-8dfa-dd8a9145be40" (UID: "25afb96c-6351-470d-8dfa-dd8a9145be40"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:13:56.484859 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.484816 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-bound-sa-token" pod "25afb96c-6351-470d-8dfa-dd8a9145be40" (UID: "25afb96c-6351-470d-8dfa-dd8a9145be40"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:13:56.485769 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.485740 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-installation-pull-secrets" pod "25afb96c-6351-470d-8dfa-dd8a9145be40" (UID: "25afb96c-6351-470d-8dfa-dd8a9145be40"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:13:56.491219 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.491192 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25afb96c-6351-470d-8dfa-dd8a9145be40-ca-trust-extracted" pod "25afb96c-6351-470d-8dfa-dd8a9145be40" (UID: "25afb96c-6351-470d-8dfa-dd8a9145be40"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:13:56.572963 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.572926 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:56.583080 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.583054 5056 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-image-registry-private-configuration\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:56.583157 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.583081 5056 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-bound-sa-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:56.583157 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.583094 5056 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/25afb96c-6351-470d-8dfa-dd8a9145be40-ca-trust-extracted\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:56.583157 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.583103 5056 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-trusted-ca\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:56.583157 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.583114 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzc9p\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-kube-api-access-bzc9p\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:56.583157 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.583123 5056 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/25afb96c-6351-470d-8dfa-dd8a9145be40-installation-pull-secrets\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:56.583157 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.583131 5056 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-tls\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:56.583157 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:56.583143 5056 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/25afb96c-6351-470d-8dfa-dd8a9145be40-registry-certificates\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:13:57.192822 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:57.192783 5056 generic.go:356] "Generic (PLEG): container finished" podID="25afb96c-6351-470d-8dfa-dd8a9145be40" containerID="17bd8fa27348541385b4a3db0c1fccaadd53c1bff34b99bf555b8f9c3405c5a1" exitCode=0 Oct 29 23:13:57.192822 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:57.192821 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" Oct 29 23:13:57.193330 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:57.192851 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" event={"ID":"25afb96c-6351-470d-8dfa-dd8a9145be40","Type":"ContainerDied","Data":"17bd8fa27348541385b4a3db0c1fccaadd53c1bff34b99bf555b8f9c3405c5a1"} Oct 29 23:13:57.193330 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:57.192905 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cbdc4b876-9xjlm" event={"ID":"25afb96c-6351-470d-8dfa-dd8a9145be40","Type":"ContainerDied","Data":"c87ce8d555e9bc090049a675fd0023eb53d730a4c298de5166748ed307fa3802"} Oct 29 23:13:57.193330 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:57.192925 5056 scope.go:122] "RemoveContainer" containerID="17bd8fa27348541385b4a3db0c1fccaadd53c1bff34b99bf555b8f9c3405c5a1" Oct 29 23:13:57.201109 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:57.201093 5056 scope.go:122] "RemoveContainer" containerID="17bd8fa27348541385b4a3db0c1fccaadd53c1bff34b99bf555b8f9c3405c5a1" Oct 29 23:13:57.201407 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:13:57.201389 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17bd8fa27348541385b4a3db0c1fccaadd53c1bff34b99bf555b8f9c3405c5a1\": container with ID starting with 17bd8fa27348541385b4a3db0c1fccaadd53c1bff34b99bf555b8f9c3405c5a1 not found: ID does not exist" containerID="17bd8fa27348541385b4a3db0c1fccaadd53c1bff34b99bf555b8f9c3405c5a1" Oct 29 23:13:57.201488 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:57.201413 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17bd8fa27348541385b4a3db0c1fccaadd53c1bff34b99bf555b8f9c3405c5a1"} err="failed to get container status \"17bd8fa27348541385b4a3db0c1fccaadd53c1bff34b99bf555b8f9c3405c5a1\": rpc error: code = NotFound desc = could not find container \"17bd8fa27348541385b4a3db0c1fccaadd53c1bff34b99bf555b8f9c3405c5a1\": container with ID starting with 17bd8fa27348541385b4a3db0c1fccaadd53c1bff34b99bf555b8f9c3405c5a1 not found: ID does not exist" Oct 29 23:13:57.270233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:57.270200 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cbdc4b876-9xjlm"] Oct 29 23:13:57.283251 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:57.283223 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7cbdc4b876-9xjlm"] Oct 29 23:13:58.290976 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:58.290937 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:58.514433 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:58.514399 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:13:58.768320 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:13:58.768289 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25afb96c-6351-470d-8dfa-dd8a9145be40" path="/var/lib/kubelet/pods/25afb96c-6351-470d-8dfa-dd8a9145be40/volumes" Oct 29 23:14:02.128803 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:02.128767 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:14:02.359541 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:02.359506 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:14:02.973697 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:02.973655 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:14:03.176613 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:03.176579 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:14:05.765612 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:05.765578 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:14:05.924983 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:05.924940 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:14:11.085039 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:11.085004 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:14:11.262373 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:11.262337 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:14:17.277301 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:17.277252 5056 generic.go:356] "Generic (PLEG): container finished" podID="8f11a401-ced0-47dc-8814-c23849ba794b" containerID="14bd2f8d9b7b87ac8e1a34b97d52be9fa02320b64500d0a6834d21ae65b7d9fb" exitCode=0 Oct 29 23:14:17.277785 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:17.277331 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" event={"ID":"8f11a401-ced0-47dc-8814-c23849ba794b","Type":"ContainerDied","Data":"14bd2f8d9b7b87ac8e1a34b97d52be9fa02320b64500d0a6834d21ae65b7d9fb"} Oct 29 23:14:17.277785 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:17.277740 5056 scope.go:122] "RemoveContainer" containerID="14bd2f8d9b7b87ac8e1a34b97d52be9fa02320b64500d0a6834d21ae65b7d9fb" Oct 29 23:14:18.282540 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:18.282506 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-mc79c" event={"ID":"8f11a401-ced0-47dc-8814-c23849ba794b","Type":"ContainerStarted","Data":"d1415373d34509323bb2c5d07f1aab9a5e19f96999621f8e2d01262118bf981e"} Oct 29 23:14:20.712702 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:20.712668 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:14:20.875521 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:20.875489 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:14:22.304809 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:22.304774 5056 generic.go:356] "Generic (PLEG): container finished" podID="837e8971-01d7-44f5-81bb-bf206d191f2d" containerID="de99a0ae73dca6e7394428edade9d95365eb9c5224a7af6940d5e72610e7e6d8" exitCode=0 Oct 29 23:14:22.305205 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:22.304836 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" event={"ID":"837e8971-01d7-44f5-81bb-bf206d191f2d","Type":"ContainerDied","Data":"de99a0ae73dca6e7394428edade9d95365eb9c5224a7af6940d5e72610e7e6d8"} Oct 29 23:14:22.305205 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:22.305143 5056 scope.go:122] "RemoveContainer" containerID="de99a0ae73dca6e7394428edade9d95365eb9c5224a7af6940d5e72610e7e6d8" Oct 29 23:14:22.418619 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:14:22.418579 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:14:23.310207 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:23.310173 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-b9jwd" event={"ID":"837e8971-01d7-44f5-81bb-bf206d191f2d","Type":"ContainerStarted","Data":"6263ddcc52f6beb15f104676f605126fdac7d1c464839b9e86b5c3a5652eac40"} Oct 29 23:14:35.362874 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:35.362778 5056 generic.go:356] "Generic (PLEG): container finished" podID="a7bc8c39-796f-40df-bca0-eddc3eab3116" containerID="bb99e9b0eda2fbe2dbbf94cd6a9639ec8286829091abacf6833515001f4fc3af" exitCode=0 Oct 29 23:14:35.362874 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:35.362850 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" event={"ID":"a7bc8c39-796f-40df-bca0-eddc3eab3116","Type":"ContainerDied","Data":"bb99e9b0eda2fbe2dbbf94cd6a9639ec8286829091abacf6833515001f4fc3af"} Oct 29 23:14:36.510664 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:36.510635 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" Oct 29 23:14:36.604759 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:36.604708 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7bc8c39-796f-40df-bca0-eddc3eab3116-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7bc8c39-796f-40df-bca0-eddc3eab3116-service-ca-bundle\") pod \"a7bc8c39-796f-40df-bca0-eddc3eab3116\" (UID: \"a7bc8c39-796f-40df-bca0-eddc3eab3116\") " Oct 29 23:14:36.604759 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:36.604778 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7bc8c39-796f-40df-bca0-eddc3eab3116-kube-api-access-c6v5s\" (UniqueName: \"kubernetes.io/projected/a7bc8c39-796f-40df-bca0-eddc3eab3116-kube-api-access-c6v5s\") pod \"a7bc8c39-796f-40df-bca0-eddc3eab3116\" (UID: \"a7bc8c39-796f-40df-bca0-eddc3eab3116\") " Oct 29 23:14:36.605023 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:36.604801 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7bc8c39-796f-40df-bca0-eddc3eab3116-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7bc8c39-796f-40df-bca0-eddc3eab3116-trusted-ca-bundle\") pod \"a7bc8c39-796f-40df-bca0-eddc3eab3116\" (UID: \"a7bc8c39-796f-40df-bca0-eddc3eab3116\") " Oct 29 23:14:36.605023 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:36.604839 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7bc8c39-796f-40df-bca0-eddc3eab3116-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a7bc8c39-796f-40df-bca0-eddc3eab3116-archives-path\") pod \"a7bc8c39-796f-40df-bca0-eddc3eab3116\" (UID: \"a7bc8c39-796f-40df-bca0-eddc3eab3116\") " Oct 29 23:14:36.605180 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:36.605157 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7bc8c39-796f-40df-bca0-eddc3eab3116-service-ca-bundle" pod "a7bc8c39-796f-40df-bca0-eddc3eab3116" (UID: "a7bc8c39-796f-40df-bca0-eddc3eab3116"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:14:36.605240 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:36.605183 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7bc8c39-796f-40df-bca0-eddc3eab3116-archives-path" pod "a7bc8c39-796f-40df-bca0-eddc3eab3116" (UID: "a7bc8c39-796f-40df-bca0-eddc3eab3116"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:14:36.605240 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:36.605160 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7bc8c39-796f-40df-bca0-eddc3eab3116-trusted-ca-bundle" pod "a7bc8c39-796f-40df-bca0-eddc3eab3116" (UID: "a7bc8c39-796f-40df-bca0-eddc3eab3116"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:14:36.606884 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:36.606860 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7bc8c39-796f-40df-bca0-eddc3eab3116-kube-api-access-c6v5s" pod "a7bc8c39-796f-40df-bca0-eddc3eab3116" (UID: "a7bc8c39-796f-40df-bca0-eddc3eab3116"). InnerVolumeSpecName "kube-api-access-c6v5s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:14:36.705852 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:36.705802 5056 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7bc8c39-796f-40df-bca0-eddc3eab3116-service-ca-bundle\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:14:36.705852 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:36.705850 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6v5s\" (UniqueName: \"kubernetes.io/projected/a7bc8c39-796f-40df-bca0-eddc3eab3116-kube-api-access-c6v5s\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:14:36.705852 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:36.705860 5056 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a7bc8c39-796f-40df-bca0-eddc3eab3116-trusted-ca-bundle\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:14:36.705852 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:36.705869 5056 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/a7bc8c39-796f-40df-bca0-eddc3eab3116-archives-path\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:14:37.372345 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:37.372318 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" Oct 29 23:14:37.372345 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:37.372333 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-mhdnw-4d4k4" event={"ID":"a7bc8c39-796f-40df-bca0-eddc3eab3116","Type":"ContainerDied","Data":"1494453991a43a5b6f5c4117a0b4c541ed0ddbfd0848d19f6036ed8d35f82aa7"} Oct 29 23:14:37.372677 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:37.372386 5056 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1494453991a43a5b6f5c4117a0b4c541ed0ddbfd0848d19f6036ed8d35f82aa7" Oct 29 23:14:37.872234 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:37.872184 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:14:38.030839 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:38.030799 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:14:41.662769 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:41.662695 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:14:41.819927 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:41.819885 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:14:42.540504 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:42.540471 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:14:52.414320 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:14:52.414281 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:14:57.728552 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:57.728510 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:14:57.887975 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:57.887933 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:14:58.341228 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:14:58.341196 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:15:22.414129 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:15:22.414087 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:15:42.663598 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:15:42.663557 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:15:52.414080 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:15:52.414043 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:16:22.413945 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:16:22.413908 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:16:42.781477 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:16:42.781403 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:16:44.447841 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:16:44.447799 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:16:44.609070 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:16:44.609033 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:16:52.413954 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:16:52.413918 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:17:01.746073 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:17:01.746039 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:17:05.712172 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:17:05.712139 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:17:05.876736 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:17:05.876706 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:17:06.765957 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:17:06.765916 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:17:22.414829 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:17:22.414785 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:17:42.908275 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:17:42.908227 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:17:52.414370 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:17:52.414332 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:17:52.778744 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:17:52.778666 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:17:52.787201 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:17:52.787174 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:18:22.413909 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:18:22.413868 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:18:43.038231 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:18:43.038192 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:18:52.413745 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:18:52.413699 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:18:56.540313 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:18:56.540249 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:18:56.712838 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:18:56.712804 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:10.357552 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:10.357515 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:18.697391 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:18.697354 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:20.085837 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:20.085800 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:22.131779 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:22.131743 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:22.305025 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:22.304993 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:22.414056 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:19:22.414019 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:19:23.740417 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:23.740385 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:23.902912 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:23.902877 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:24.275756 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:24.275723 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:24.460278 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:24.460220 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:25.572761 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:25.572727 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:26.132030 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:26.131996 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:28.879327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:28.879296 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:29.030898 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:29.030861 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:33.006410 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:33.006372 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:33.179899 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:33.179864 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:34.763382 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:34.763301 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:36.989379 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:36.989341 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:37.140992 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:37.140959 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:43.164830 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:43.164794 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:43.763711 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:43.763679 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:43.929365 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:43.929335 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:45.355442 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:45.355411 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:51.850481 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:51.850441 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:52.003745 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:19:52.003711 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:19:52.414598 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:19:52.414563 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:20:00.127693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:00.127653 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:00.520216 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:00.520183 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:00.933143 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:00.933108 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:01.025546 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.025518 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4"] Oct 29 23:20:01.025831 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.025818 5056 cpu_manager.go:414] "containerMap: removing container" podUID="25afb96c-6351-470d-8dfa-dd8a9145be40" containerName="registry" Oct 29 23:20:01.025890 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.025832 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25afb96c-6351-470d-8dfa-dd8a9145be40" containerName="registry" Oct 29 23:20:01.025890 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.025852 5056 cpu_manager.go:414] "containerMap: removing container" podUID="fce92e88-114c-49c7-b658-193d508126e9" containerName="insights-gathering" Oct 29 23:20:01.025890 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.025857 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fce92e88-114c-49c7-b658-193d508126e9" containerName="insights-gathering" Oct 29 23:20:01.025890 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.025871 5056 cpu_manager.go:414] "containerMap: removing container" podUID="a7bc8c39-796f-40df-bca0-eddc3eab3116" containerName="insights-gathering" Oct 29 23:20:01.025890 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.025876 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7bc8c39-796f-40df-bca0-eddc3eab3116" containerName="insights-gathering" Oct 29 23:20:01.025890 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.025891 5056 cpu_manager.go:414] "containerMap: removing container" podUID="6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" containerName="registry" Oct 29 23:20:01.026085 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.025897 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" containerName="registry" Oct 29 23:20:01.026085 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.025947 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7bc8c39-796f-40df-bca0-eddc3eab3116" containerName="insights-gathering" Oct 29 23:20:01.026085 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.025956 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d3d6fb4-8dac-4983-9fa1-84fb5b1b5a6c" containerName="registry" Oct 29 23:20:01.026085 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.025964 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="25afb96c-6351-470d-8dfa-dd8a9145be40" containerName="registry" Oct 29 23:20:01.026085 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.025971 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="fce92e88-114c-49c7-b658-193d508126e9" containerName="insights-gathering" Oct 29 23:20:01.029094 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.029068 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.056181 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.056155 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Oct 29 23:20:01.060463 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.060446 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-295kq\"" Oct 29 23:20:01.060517 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.060497 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Oct 29 23:20:01.060557 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.060506 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Oct 29 23:20:01.060557 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.060522 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-metrics-tls\"" Oct 29 23:20:01.071254 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.071236 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Oct 29 23:20:01.110863 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.110836 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4"] Oct 29 23:20:01.118902 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.118876 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.119008 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.118981 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-manifests\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.119046 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.119017 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-webhook-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.119084 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.119047 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-tmp\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.119162 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.119147 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-metrics-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.119197 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.119170 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8h7j\" (UniqueName: \"kubernetes.io/projected/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-kube-api-access-l8h7j\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.219859 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.219761 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-metrics-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.219859 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.219802 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8h7j\" (UniqueName: \"kubernetes.io/projected/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-kube-api-access-l8h7j\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.219859 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.219836 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.220465 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.219880 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-manifests\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.220465 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.219918 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-webhook-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.220465 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.219957 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-tmp\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.220594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.220462 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-manifests\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.222280 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.222241 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-tmp\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.222627 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.222599 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.222861 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.222840 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-webhook-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.222904 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.222839 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-metrics-cert\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.244631 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.244606 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8h7j\" (UniqueName: \"kubernetes.io/projected/b79d1232-88bc-4d1d-8a7c-78cc858b36f5-kube-api-access-l8h7j\") pod \"opendatahub-operator-controller-manager-57666cc899-7msc4\" (UID: \"b79d1232-88bc-4d1d-8a7c-78cc858b36f5\") " pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.284820 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.284791 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:01.343806 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.343773 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:01.430385 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.430353 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:01.616221 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.616152 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:01.685830 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.685806 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4"] Oct 29 23:20:01.687926 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:20:01.687900 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb79d1232_88bc_4d1d_8a7c_78cc858b36f5.slice/crio-3f3b176a9c706544013ab94dd94e1a4ea047eab62482261057627949c6a936f8 WatchSource:0}: Error finding container 3f3b176a9c706544013ab94dd94e1a4ea047eab62482261057627949c6a936f8: Status 404 returned error can't find the container with id 3f3b176a9c706544013ab94dd94e1a4ea047eab62482261057627949c6a936f8 Oct 29 23:20:01.689453 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.689430 5056 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 29 23:20:01.792196 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:01.792163 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:02.450115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:02.450071 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:02.612698 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:02.612661 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:02.671127 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:02.671046 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" event={"ID":"b79d1232-88bc-4d1d-8a7c-78cc858b36f5","Type":"ContainerStarted","Data":"3f3b176a9c706544013ab94dd94e1a4ea047eab62482261057627949c6a936f8"} Oct 29 23:20:04.681279 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:04.681233 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" event={"ID":"b79d1232-88bc-4d1d-8a7c-78cc858b36f5","Type":"ContainerStarted","Data":"d093137b52a75db3dfb9e9b68f441065871310fe729ceb3168d06a90117735f6"} Oct 29 23:20:05.686523 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:05.686482 5056 generic.go:356] "Generic (PLEG): container finished" podID="b79d1232-88bc-4d1d-8a7c-78cc858b36f5" containerID="d093137b52a75db3dfb9e9b68f441065871310fe729ceb3168d06a90117735f6" exitCode=0 Oct 29 23:20:05.686985 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:05.686566 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" event={"ID":"b79d1232-88bc-4d1d-8a7c-78cc858b36f5","Type":"ContainerDied","Data":"d093137b52a75db3dfb9e9b68f441065871310fe729ceb3168d06a90117735f6"} Oct 29 23:20:06.692688 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:06.692657 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" event={"ID":"b79d1232-88bc-4d1d-8a7c-78cc858b36f5","Type":"ContainerStarted","Data":"702444d7b406bf0ddebbb6a408be34a67fd40fc7f9e6552be80eb5d2501555dd"} Oct 29 23:20:06.693100 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:06.692769 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:06.784394 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:06.784343 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" podStartSLOduration=3.191438916 podStartE2EDuration="6.784329977s" totalImagesPullingTime="3.156692632s" totalInitContainerRuntime="436.198429ms" isStatefulPod=false podCreationTimestamp="2026-10-29 23:20:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:20:06.771213565 +0000 UTC m=+434.137961197" watchObservedRunningTime="2026-10-29 23:20:06.784329977 +0000 UTC m=+434.151077608" Oct 29 23:20:17.699195 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:17.699161 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-57666cc899-7msc4" Oct 29 23:20:20.255253 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:20.255217 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:20.503844 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:20.503804 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:22.414176 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:20:22.414133 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:20:23.767555 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:23.767518 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:23.987138 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:23.987109 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/odh-observability-584f99c788-bbcsr"] Oct 29 23:20:23.996009 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:23.995979 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-observability-584f99c788-bbcsr" Oct 29 23:20:24.014951 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:24.014923 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzrnd\" (UniqueName: \"kubernetes.io/projected/9607303a-926f-43a9-b17b-c7666e93321e-kube-api-access-kzrnd\") pod \"odh-observability-584f99c788-bbcsr\" (UID: \"9607303a-926f-43a9-b17b-c7666e93321e\") " pod="opendatahub/odh-observability-584f99c788-bbcsr" Oct 29 23:20:24.015067 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:24.014989 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/9607303a-926f-43a9-b17b-c7666e93321e-webhook-certs\") pod \"odh-observability-584f99c788-bbcsr\" (UID: \"9607303a-926f-43a9-b17b-c7666e93321e\") " pod="opendatahub/odh-observability-584f99c788-bbcsr" Oct 29 23:20:24.025633 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:24.025562 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-observability-webhook-cert\"" Oct 29 23:20:24.025633 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:24.025580 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-observability-dockercfg-bxbcn\"" Oct 29 23:20:24.027518 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:24.027495 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-observability-584f99c788-bbcsr"] Oct 29 23:20:24.105679 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:24.105643 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:24.115946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:24.115917 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzrnd\" (UniqueName: \"kubernetes.io/projected/9607303a-926f-43a9-b17b-c7666e93321e-kube-api-access-kzrnd\") pod \"odh-observability-584f99c788-bbcsr\" (UID: \"9607303a-926f-43a9-b17b-c7666e93321e\") " pod="opendatahub/odh-observability-584f99c788-bbcsr" Oct 29 23:20:24.116074 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:24.115968 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/9607303a-926f-43a9-b17b-c7666e93321e-webhook-certs\") pod \"odh-observability-584f99c788-bbcsr\" (UID: \"9607303a-926f-43a9-b17b-c7666e93321e\") " pod="opendatahub/odh-observability-584f99c788-bbcsr" Oct 29 23:20:24.116138 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:20:24.116123 5056 secret.go:189] Couldn't get secret opendatahub/odh-observability-webhook-cert: secret "odh-observability-webhook-cert" not found Oct 29 23:20:24.116199 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:20:24.116189 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9607303a-926f-43a9-b17b-c7666e93321e-webhook-certs podName:9607303a-926f-43a9-b17b-c7666e93321e nodeName:}" failed. No retries permitted until 2026-10-29 23:20:24.616172378 +0000 UTC m=+451.982920004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-certs" (UniqueName: "kubernetes.io/secret/9607303a-926f-43a9-b17b-c7666e93321e-webhook-certs") pod "odh-observability-584f99c788-bbcsr" (UID: "9607303a-926f-43a9-b17b-c7666e93321e") : secret "odh-observability-webhook-cert" not found Oct 29 23:20:24.140725 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:24.140701 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzrnd\" (UniqueName: \"kubernetes.io/projected/9607303a-926f-43a9-b17b-c7666e93321e-kube-api-access-kzrnd\") pod \"odh-observability-584f99c788-bbcsr\" (UID: \"9607303a-926f-43a9-b17b-c7666e93321e\") " pod="opendatahub/odh-observability-584f99c788-bbcsr" Oct 29 23:20:24.571258 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:24.571221 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:24.620545 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:24.620510 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/9607303a-926f-43a9-b17b-c7666e93321e-webhook-certs\") pod \"odh-observability-584f99c788-bbcsr\" (UID: \"9607303a-926f-43a9-b17b-c7666e93321e\") " pod="opendatahub/odh-observability-584f99c788-bbcsr" Oct 29 23:20:24.622913 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:24.622883 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-certs\" (UniqueName: \"kubernetes.io/secret/9607303a-926f-43a9-b17b-c7666e93321e-webhook-certs\") pod \"odh-observability-584f99c788-bbcsr\" (UID: \"9607303a-926f-43a9-b17b-c7666e93321e\") " pod="opendatahub/odh-observability-584f99c788-bbcsr" Oct 29 23:20:24.908126 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:24.908076 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-observability-584f99c788-bbcsr" Oct 29 23:20:25.205849 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:25.205826 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-observability-584f99c788-bbcsr"] Oct 29 23:20:25.207969 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:20:25.207940 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9607303a_926f_43a9_b17b_c7666e93321e.slice/crio-5abbc1051e2ef8568bc8ef16cba144b9c52c176058b2b0f5ec298811a1f3e98e WatchSource:0}: Error finding container 5abbc1051e2ef8568bc8ef16cba144b9c52c176058b2b0f5ec298811a1f3e98e: Status 404 returned error can't find the container with id 5abbc1051e2ef8568bc8ef16cba144b9c52c176058b2b0f5ec298811a1f3e98e Oct 29 23:20:25.771420 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:25.771381 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-observability-584f99c788-bbcsr" event={"ID":"9607303a-926f-43a9-b17b-c7666e93321e","Type":"ContainerStarted","Data":"5abbc1051e2ef8568bc8ef16cba144b9c52c176058b2b0f5ec298811a1f3e98e"} Oct 29 23:20:27.780962 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:27.780869 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-observability-584f99c788-bbcsr" event={"ID":"9607303a-926f-43a9-b17b-c7666e93321e","Type":"ContainerStarted","Data":"c373f0647d68980e02cba403cbd5dd54953b96e296640b73f722da7713b3b808"} Oct 29 23:20:27.781334 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:27.780991 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-observability-584f99c788-bbcsr" Oct 29 23:20:27.823804 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:27.823745 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/odh-observability-584f99c788-bbcsr" podStartSLOduration=2.609052172 podStartE2EDuration="4.823732447s" totalImagesPullingTime="2.214680275s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:20:27.823177129 +0000 UTC m=+455.189924760" watchObservedRunningTime="2026-10-29 23:20:27.823732447 +0000 UTC m=+455.190480077" Oct 29 23:20:31.786518 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:31.786487 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-8578b44655-zxqwh"] Oct 29 23:20:31.789632 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:31.789615 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8578b44655-zxqwh" Oct 29 23:20:31.807369 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:31.807343 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Oct 29 23:20:31.819877 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:31.819850 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Oct 29 23:20:31.820001 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:31.819920 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-jwvmc\"" Oct 29 23:20:31.821285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:31.821250 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8578b44655-zxqwh"] Oct 29 23:20:31.838962 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:31.838933 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:31.877108 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:31.877076 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjp7m\" (UniqueName: \"kubernetes.io/projected/bca57086-0196-48e5-ace2-92dbfa7bb5f6-kube-api-access-xjp7m\") pod \"kube-auth-proxy-8578b44655-zxqwh\" (UID: \"bca57086-0196-48e5-ace2-92dbfa7bb5f6\") " pod="openshift-ingress/kube-auth-proxy-8578b44655-zxqwh" Oct 29 23:20:31.877245 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:31.877116 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bca57086-0196-48e5-ace2-92dbfa7bb5f6-tls-certs\") pod \"kube-auth-proxy-8578b44655-zxqwh\" (UID: \"bca57086-0196-48e5-ace2-92dbfa7bb5f6\") " pod="openshift-ingress/kube-auth-proxy-8578b44655-zxqwh" Oct 29 23:20:31.877245 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:31.877215 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bca57086-0196-48e5-ace2-92dbfa7bb5f6-tmp\") pod \"kube-auth-proxy-8578b44655-zxqwh\" (UID: \"bca57086-0196-48e5-ace2-92dbfa7bb5f6\") " pod="openshift-ingress/kube-auth-proxy-8578b44655-zxqwh" Oct 29 23:20:31.977723 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:31.977690 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bca57086-0196-48e5-ace2-92dbfa7bb5f6-tls-certs\") pod \"kube-auth-proxy-8578b44655-zxqwh\" (UID: \"bca57086-0196-48e5-ace2-92dbfa7bb5f6\") " pod="openshift-ingress/kube-auth-proxy-8578b44655-zxqwh" Oct 29 23:20:31.977723 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:31.977724 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bca57086-0196-48e5-ace2-92dbfa7bb5f6-tmp\") pod \"kube-auth-proxy-8578b44655-zxqwh\" (UID: \"bca57086-0196-48e5-ace2-92dbfa7bb5f6\") " pod="openshift-ingress/kube-auth-proxy-8578b44655-zxqwh" Oct 29 23:20:31.977969 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:31.977794 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjp7m\" (UniqueName: \"kubernetes.io/projected/bca57086-0196-48e5-ace2-92dbfa7bb5f6-kube-api-access-xjp7m\") pod \"kube-auth-proxy-8578b44655-zxqwh\" (UID: \"bca57086-0196-48e5-ace2-92dbfa7bb5f6\") " pod="openshift-ingress/kube-auth-proxy-8578b44655-zxqwh" Oct 29 23:20:31.980070 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:31.980045 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bca57086-0196-48e5-ace2-92dbfa7bb5f6-tmp\") pod \"kube-auth-proxy-8578b44655-zxqwh\" (UID: \"bca57086-0196-48e5-ace2-92dbfa7bb5f6\") " pod="openshift-ingress/kube-auth-proxy-8578b44655-zxqwh" Oct 29 23:20:31.980323 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:31.980304 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bca57086-0196-48e5-ace2-92dbfa7bb5f6-tls-certs\") pod \"kube-auth-proxy-8578b44655-zxqwh\" (UID: \"bca57086-0196-48e5-ace2-92dbfa7bb5f6\") " pod="openshift-ingress/kube-auth-proxy-8578b44655-zxqwh" Oct 29 23:20:32.014070 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:32.014040 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjp7m\" (UniqueName: \"kubernetes.io/projected/bca57086-0196-48e5-ace2-92dbfa7bb5f6-kube-api-access-xjp7m\") pod \"kube-auth-proxy-8578b44655-zxqwh\" (UID: \"bca57086-0196-48e5-ace2-92dbfa7bb5f6\") " pod="openshift-ingress/kube-auth-proxy-8578b44655-zxqwh" Oct 29 23:20:32.077192 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:32.077114 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:32.101758 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:32.101738 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-8578b44655-zxqwh" Oct 29 23:20:32.397219 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:32.397144 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-8578b44655-zxqwh"] Oct 29 23:20:32.399423 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:20:32.399396 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbca57086_0196_48e5_ace2_92dbfa7bb5f6.slice/crio-e410bdf3911d8ee7895f1d84aed78adac0ef95a3d1f21b902c3e190985446071 WatchSource:0}: Error finding container e410bdf3911d8ee7895f1d84aed78adac0ef95a3d1f21b902c3e190985446071: Status 404 returned error can't find the container with id e410bdf3911d8ee7895f1d84aed78adac0ef95a3d1f21b902c3e190985446071 Oct 29 23:20:32.802121 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:32.802037 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8578b44655-zxqwh" event={"ID":"bca57086-0196-48e5-ace2-92dbfa7bb5f6","Type":"ContainerStarted","Data":"e410bdf3911d8ee7895f1d84aed78adac0ef95a3d1f21b902c3e190985446071"} Oct 29 23:20:33.500602 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.500566 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg"] Oct 29 23:20:33.505016 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.504991 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.520977 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.520956 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Oct 29 23:20:33.523456 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.523432 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Oct 29 23:20:33.527450 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.527428 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Oct 29 23:20:33.528428 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.528409 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Oct 29 23:20:33.533845 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.533819 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bbcdn\"" Oct 29 23:20:33.535616 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.535595 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg"] Oct 29 23:20:33.564499 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.564471 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:33.591855 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.591816 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.591998 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.591863 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2286415b-cca3-49fc-8a53-7c2de7038734-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.591998 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.591895 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2286415b-cca3-49fc-8a53-7c2de7038734-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.592102 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.592031 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6l7n\" (UniqueName: \"kubernetes.io/projected/2286415b-cca3-49fc-8a53-7c2de7038734-kube-api-access-d6l7n\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.592102 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.592071 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.592102 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.592091 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.592237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.592146 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2286415b-cca3-49fc-8a53-7c2de7038734-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.693327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.693288 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6l7n\" (UniqueName: \"kubernetes.io/projected/2286415b-cca3-49fc-8a53-7c2de7038734-kube-api-access-d6l7n\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.693507 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.693374 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.693507 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.693410 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.693685 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.693660 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2286415b-cca3-49fc-8a53-7c2de7038734-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.693739 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.693727 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.693784 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.693756 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2286415b-cca3-49fc-8a53-7c2de7038734-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.693826 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.693790 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2286415b-cca3-49fc-8a53-7c2de7038734-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.694704 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.694633 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2286415b-cca3-49fc-8a53-7c2de7038734-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.696487 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.696459 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.696713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.696678 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.696800 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.696709 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2286415b-cca3-49fc-8a53-7c2de7038734-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.697030 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.697002 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.730177 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.730153 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2286415b-cca3-49fc-8a53-7c2de7038734-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.730614 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.730593 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6l7n\" (UniqueName: \"kubernetes.io/projected/2286415b-cca3-49fc-8a53-7c2de7038734-kube-api-access-d6l7n\") pod \"istiod-openshift-gateway-7876bfcb74-g6mwg\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:33.777004 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.776926 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:33.820903 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:33.820865 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:34.147497 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:34.147472 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg"] Oct 29 23:20:34.149167 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:20:34.149141 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2286415b_cca3_49fc_8a53_7c2de7038734.slice/crio-2ad0a070d0bf0141a4f0759ecfb0cb1fc44f2d59c80cbf8beaa2167f333cc7f9 WatchSource:0}: Error finding container 2ad0a070d0bf0141a4f0759ecfb0cb1fc44f2d59c80cbf8beaa2167f333cc7f9: Status 404 returned error can't find the container with id 2ad0a070d0bf0141a4f0759ecfb0cb1fc44f2d59c80cbf8beaa2167f333cc7f9 Oct 29 23:20:34.812433 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:34.812395 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" event={"ID":"2286415b-cca3-49fc-8a53-7c2de7038734","Type":"ContainerStarted","Data":"2ad0a070d0bf0141a4f0759ecfb0cb1fc44f2d59c80cbf8beaa2167f333cc7f9"} Oct 29 23:20:34.813973 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:34.813948 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-8578b44655-zxqwh" event={"ID":"bca57086-0196-48e5-ace2-92dbfa7bb5f6","Type":"ContainerStarted","Data":"ecd63273bfc98cd9bc68dbe662df12f754fceb62013641f80cee4727269f67de"} Oct 29 23:20:34.856374 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:34.856328 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-8578b44655-zxqwh" podStartSLOduration=1.528851334 podStartE2EDuration="3.856314687s" totalImagesPullingTime="2.327463353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:20:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:20:34.855942519 +0000 UTC m=+462.222690149" watchObservedRunningTime="2026-10-29 23:20:34.856314687 +0000 UTC m=+462.223062318" Oct 29 23:20:34.960982 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:34.960937 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:36.538974 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:36.538925 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:36.731580 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:36.731528 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:37.012874 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:37.012840 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:20:37.012990 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:37.012905 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:20:37.805475 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:37.805429 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:37.828732 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:37.828699 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" event={"ID":"2286415b-cca3-49fc-8a53-7c2de7038734","Type":"ContainerStarted","Data":"1b3809aea5a0c74e90a94fed733aaadabc6dc1d333289d8d27a0690c86587b0b"} Oct 29 23:20:37.828904 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:37.828827 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:37.911830 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:37.911767 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" podStartSLOduration=2.05159549 podStartE2EDuration="4.911750986s" totalImagesPullingTime="2.860155496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:20:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:20:37.89774608 +0000 UTC m=+465.264493710" watchObservedRunningTime="2026-10-29 23:20:37.911750986 +0000 UTC m=+465.278498617" Oct 29 23:20:38.039844 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:38.039797 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:38.786944 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:38.786917 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-observability-584f99c788-bbcsr" Oct 29 23:20:38.834542 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:38.834511 5056 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-g6mwg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:20:38.834898 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:38.834559 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" podUID="2286415b-cca3-49fc-8a53-7c2de7038734" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:20:39.185351 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:39.185319 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:39.346694 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:39.346660 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:39.546467 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:39.546383 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:41.834592 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:41.834564 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:20:43.252969 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:43.252936 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:51.741192 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:51.741156 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:51.895606 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:20:51.895570 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:20:52.414853 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:20:52.414818 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:21:15.255925 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:15.255822 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:21:15.415809 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:15.415778 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:21:22.414238 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:21:22.414200 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:21:23.539618 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.539581 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p"] Oct 29 23:21:23.548670 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.548648 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.580563 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.580536 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p"] Oct 29 23:21:23.633617 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.633585 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/712e0b41-1ea6-44f3-8021-f619394ecd34-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.633617 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.633621 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/712e0b41-1ea6-44f3-8021-f619394ecd34-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.633828 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.633717 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/712e0b41-1ea6-44f3-8021-f619394ecd34-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.633828 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.633746 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/712e0b41-1ea6-44f3-8021-f619394ecd34-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.633828 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.633772 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86c25\" (UniqueName: \"kubernetes.io/projected/712e0b41-1ea6-44f3-8021-f619394ecd34-kube-api-access-86c25\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.633991 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.633916 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/712e0b41-1ea6-44f3-8021-f619394ecd34-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.634032 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.634009 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/712e0b41-1ea6-44f3-8021-f619394ecd34-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.734890 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.734856 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/712e0b41-1ea6-44f3-8021-f619394ecd34-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.734890 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.734893 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/712e0b41-1ea6-44f3-8021-f619394ecd34-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.735128 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.734927 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/712e0b41-1ea6-44f3-8021-f619394ecd34-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.735128 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.734945 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/712e0b41-1ea6-44f3-8021-f619394ecd34-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.735128 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.734964 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86c25\" (UniqueName: \"kubernetes.io/projected/712e0b41-1ea6-44f3-8021-f619394ecd34-kube-api-access-86c25\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.735128 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.734994 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/712e0b41-1ea6-44f3-8021-f619394ecd34-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.735128 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.735052 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/712e0b41-1ea6-44f3-8021-f619394ecd34-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.735877 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.735843 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/712e0b41-1ea6-44f3-8021-f619394ecd34-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.737315 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.737292 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/712e0b41-1ea6-44f3-8021-f619394ecd34-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.737548 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.737531 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/712e0b41-1ea6-44f3-8021-f619394ecd34-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.737593 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.737529 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/712e0b41-1ea6-44f3-8021-f619394ecd34-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.737629 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.737595 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/712e0b41-1ea6-44f3-8021-f619394ecd34-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.759111 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.759083 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/712e0b41-1ea6-44f3-8021-f619394ecd34-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.759332 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.759313 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86c25\" (UniqueName: \"kubernetes.io/projected/712e0b41-1ea6-44f3-8021-f619394ecd34-kube-api-access-86c25\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2dl5p\" (UID: \"712e0b41-1ea6-44f3-8021-f619394ecd34\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:23.859709 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:23.859645 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:24.174980 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:24.174953 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p"] Oct 29 23:21:24.177299 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:21:24.177272 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod712e0b41_1ea6_44f3_8021_f619394ecd34.slice/crio-0d2a92d6d2ec87ab448e4e426ce6690cb1591343c4132dafa036746a896cb51c WatchSource:0}: Error finding container 0d2a92d6d2ec87ab448e4e426ce6690cb1591343c4132dafa036746a896cb51c: Status 404 returned error can't find the container with id 0d2a92d6d2ec87ab448e4e426ce6690cb1591343c4132dafa036746a896cb51c Oct 29 23:21:24.179291 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:24.179237 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:21:24.179378 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:24.179318 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:21:24.460507 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:24.460438 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:21:24.640443 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:24.640406 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:21:24.818958 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:24.818879 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:21:25.026981 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:25.026944 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" event={"ID":"712e0b41-1ea6-44f3-8021-f619394ecd34","Type":"ContainerStarted","Data":"5478dd596e7df95a41c2bd397be6c4e4431281e5a58efb27b3ecfbef0ff53c42"} Oct 29 23:21:25.027158 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:25.026996 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" event={"ID":"712e0b41-1ea6-44f3-8021-f619394ecd34","Type":"ContainerStarted","Data":"0d2a92d6d2ec87ab448e4e426ce6690cb1591343c4132dafa036746a896cb51c"} Oct 29 23:21:25.027158 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:25.027111 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:25.028715 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:25.028689 5056 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-2dl5p container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:21:25.028821 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:25.028735 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" podUID="712e0b41-1ea6-44f3-8021-f619394ecd34" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:21:25.051972 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:25.051947 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:21:25.101085 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:25.101001 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" podStartSLOduration=2.100989199 podStartE2EDuration="2.100989199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:21:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:21:25.087356678 +0000 UTC m=+512.454104310" watchObservedRunningTime="2026-10-29 23:21:25.100989199 +0000 UTC m=+512.467736830" Oct 29 23:21:25.310895 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:25.310858 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:21:26.032571 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.032540 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2dl5p" Oct 29 23:21:26.127853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.127812 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg"] Oct 29 23:21:26.128049 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.128028 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" podUID="2286415b-cca3-49fc-8a53-7c2de7038734" containerName="discovery" containerID="cri-o://1b3809aea5a0c74e90a94fed733aaadabc6dc1d333289d8d27a0690c86587b0b" gracePeriod=30 Oct 29 23:21:26.373850 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.373827 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:21:26.460520 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.460487 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2286415b-cca3-49fc-8a53-7c2de7038734-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2286415b-cca3-49fc-8a53-7c2de7038734-local-certs\") pod \"2286415b-cca3-49fc-8a53-7c2de7038734\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " Oct 29 23:21:26.460520 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.460525 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2286415b-cca3-49fc-8a53-7c2de7038734-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2286415b-cca3-49fc-8a53-7c2de7038734-istio-csr-ca-configmap\") pod \"2286415b-cca3-49fc-8a53-7c2de7038734\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " Oct 29 23:21:26.460746 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.460575 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2286415b-cca3-49fc-8a53-7c2de7038734-istio-token\" (UniqueName: \"kubernetes.io/projected/2286415b-cca3-49fc-8a53-7c2de7038734-istio-token\") pod \"2286415b-cca3-49fc-8a53-7c2de7038734\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " Oct 29 23:21:26.460746 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.460604 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-cacerts\" (UniqueName: \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-cacerts\") pod \"2286415b-cca3-49fc-8a53-7c2de7038734\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " Oct 29 23:21:26.460746 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.460634 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-istio-kubeconfig\") pod \"2286415b-cca3-49fc-8a53-7c2de7038734\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " Oct 29 23:21:26.460746 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.460664 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-istio-csr-dns-cert\") pod \"2286415b-cca3-49fc-8a53-7c2de7038734\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " Oct 29 23:21:26.460951 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.460797 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2286415b-cca3-49fc-8a53-7c2de7038734-kube-api-access-d6l7n\" (UniqueName: \"kubernetes.io/projected/2286415b-cca3-49fc-8a53-7c2de7038734-kube-api-access-d6l7n\") pod \"2286415b-cca3-49fc-8a53-7c2de7038734\" (UID: \"2286415b-cca3-49fc-8a53-7c2de7038734\") " Oct 29 23:21:26.461382 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.461029 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2286415b-cca3-49fc-8a53-7c2de7038734-istio-csr-ca-configmap" pod "2286415b-cca3-49fc-8a53-7c2de7038734" (UID: "2286415b-cca3-49fc-8a53-7c2de7038734"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:21:26.462981 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.462952 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-cacerts" pod "2286415b-cca3-49fc-8a53-7c2de7038734" (UID: "2286415b-cca3-49fc-8a53-7c2de7038734"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:21:26.463084 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.462995 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2286415b-cca3-49fc-8a53-7c2de7038734-local-certs" pod "2286415b-cca3-49fc-8a53-7c2de7038734" (UID: "2286415b-cca3-49fc-8a53-7c2de7038734"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:21:26.463221 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.463201 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-istio-csr-dns-cert" pod "2286415b-cca3-49fc-8a53-7c2de7038734" (UID: "2286415b-cca3-49fc-8a53-7c2de7038734"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:21:26.463329 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.463306 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-istio-kubeconfig" pod "2286415b-cca3-49fc-8a53-7c2de7038734" (UID: "2286415b-cca3-49fc-8a53-7c2de7038734"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:21:26.463329 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.463306 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2286415b-cca3-49fc-8a53-7c2de7038734-istio-token" pod "2286415b-cca3-49fc-8a53-7c2de7038734" (UID: "2286415b-cca3-49fc-8a53-7c2de7038734"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:21:26.463455 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.463427 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2286415b-cca3-49fc-8a53-7c2de7038734-kube-api-access-d6l7n" pod "2286415b-cca3-49fc-8a53-7c2de7038734" (UID: "2286415b-cca3-49fc-8a53-7c2de7038734"). InnerVolumeSpecName "kube-api-access-d6l7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:21:26.561799 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.561728 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2286415b-cca3-49fc-8a53-7c2de7038734-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:21:26.561799 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.561754 5056 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-cacerts\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:21:26.561799 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.561762 5056 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-istio-kubeconfig\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:21:26.561799 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.561772 5056 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/2286415b-cca3-49fc-8a53-7c2de7038734-istio-csr-dns-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:21:26.561799 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.561781 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6l7n\" (UniqueName: \"kubernetes.io/projected/2286415b-cca3-49fc-8a53-7c2de7038734-kube-api-access-d6l7n\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:21:26.561799 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.561790 5056 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/2286415b-cca3-49fc-8a53-7c2de7038734-local-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:21:26.561799 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:26.561799 5056 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/2286415b-cca3-49fc-8a53-7c2de7038734-istio-csr-ca-configmap\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:21:27.037585 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:27.037550 5056 generic.go:356] "Generic (PLEG): container finished" podID="2286415b-cca3-49fc-8a53-7c2de7038734" containerID="1b3809aea5a0c74e90a94fed733aaadabc6dc1d333289d8d27a0690c86587b0b" exitCode=0 Oct 29 23:21:27.037986 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:27.037602 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" Oct 29 23:21:27.037986 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:27.037611 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" event={"ID":"2286415b-cca3-49fc-8a53-7c2de7038734","Type":"ContainerDied","Data":"1b3809aea5a0c74e90a94fed733aaadabc6dc1d333289d8d27a0690c86587b0b"} Oct 29 23:21:27.037986 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:27.037667 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg" event={"ID":"2286415b-cca3-49fc-8a53-7c2de7038734","Type":"ContainerDied","Data":"2ad0a070d0bf0141a4f0759ecfb0cb1fc44f2d59c80cbf8beaa2167f333cc7f9"} Oct 29 23:21:27.037986 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:27.037695 5056 scope.go:122] "RemoveContainer" containerID="1b3809aea5a0c74e90a94fed733aaadabc6dc1d333289d8d27a0690c86587b0b" Oct 29 23:21:27.046228 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:27.046211 5056 scope.go:122] "RemoveContainer" containerID="1b3809aea5a0c74e90a94fed733aaadabc6dc1d333289d8d27a0690c86587b0b" Oct 29 23:21:27.046503 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:21:27.046485 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b3809aea5a0c74e90a94fed733aaadabc6dc1d333289d8d27a0690c86587b0b\": container with ID starting with 1b3809aea5a0c74e90a94fed733aaadabc6dc1d333289d8d27a0690c86587b0b not found: ID does not exist" containerID="1b3809aea5a0c74e90a94fed733aaadabc6dc1d333289d8d27a0690c86587b0b" Oct 29 23:21:27.046552 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:27.046512 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b3809aea5a0c74e90a94fed733aaadabc6dc1d333289d8d27a0690c86587b0b"} err="failed to get container status \"1b3809aea5a0c74e90a94fed733aaadabc6dc1d333289d8d27a0690c86587b0b\": rpc error: code = NotFound desc = could not find container \"1b3809aea5a0c74e90a94fed733aaadabc6dc1d333289d8d27a0690c86587b0b\": container with ID starting with 1b3809aea5a0c74e90a94fed733aaadabc6dc1d333289d8d27a0690c86587b0b not found: ID does not exist" Oct 29 23:21:27.104512 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:27.104488 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg"] Oct 29 23:21:27.117473 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:27.117454 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-g6mwg"] Oct 29 23:21:27.165372 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:27.165339 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:21:27.320939 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:27.320833 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:21:27.478434 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:27.478400 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:21:27.716526 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:27.716473 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:21:28.768513 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:28.768477 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2286415b-cca3-49fc-8a53-7c2de7038734" path="/var/lib/kubelet/pods/2286415b-cca3-49fc-8a53-7c2de7038734/volumes" Oct 29 23:21:43.391350 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:21:43.391300 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:21:52.414961 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:21:52.414919 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:22:13.505205 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:13.505167 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:13.665372 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:13.665338 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:22.414778 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:22:22.414740 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:22:24.601447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:24.601414 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:25.391615 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:25.391580 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:26.219566 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:26.219533 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:29.147709 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.147674 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4"] Oct 29 23:22:29.148091 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.148040 5056 cpu_manager.go:414] "containerMap: removing container" podUID="2286415b-cca3-49fc-8a53-7c2de7038734" containerName="discovery" Oct 29 23:22:29.148091 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.148053 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2286415b-cca3-49fc-8a53-7c2de7038734" containerName="discovery" Oct 29 23:22:29.148159 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.148127 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="2286415b-cca3-49fc-8a53-7c2de7038734" containerName="discovery" Oct 29 23:22:29.151129 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.151115 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:29.170208 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.170181 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9rhfr\"" Oct 29 23:22:29.177546 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.177524 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Oct 29 23:22:29.177771 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.177557 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Oct 29 23:22:29.185643 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.185618 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4"] Oct 29 23:22:29.301585 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.301553 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpftc\" (UniqueName: \"kubernetes.io/projected/df4dd9e1-7bd1-43f3-abf6-c8e16405b291-kube-api-access-gpftc\") pod \"kuadrant-operator-controller-manager-6b6fb56846-pj2h4\" (UID: \"df4dd9e1-7bd1-43f3-abf6-c8e16405b291\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:29.301774 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.301652 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df4dd9e1-7bd1-43f3-abf6-c8e16405b291-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-pj2h4\" (UID: \"df4dd9e1-7bd1-43f3-abf6-c8e16405b291\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:29.308405 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.308375 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:29.402426 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.402350 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df4dd9e1-7bd1-43f3-abf6-c8e16405b291-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-pj2h4\" (UID: \"df4dd9e1-7bd1-43f3-abf6-c8e16405b291\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:29.402576 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.402421 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpftc\" (UniqueName: \"kubernetes.io/projected/df4dd9e1-7bd1-43f3-abf6-c8e16405b291-kube-api-access-gpftc\") pod \"kuadrant-operator-controller-manager-6b6fb56846-pj2h4\" (UID: \"df4dd9e1-7bd1-43f3-abf6-c8e16405b291\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:29.402739 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.402720 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df4dd9e1-7bd1-43f3-abf6-c8e16405b291-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-pj2h4\" (UID: \"df4dd9e1-7bd1-43f3-abf6-c8e16405b291\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:29.433827 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.433798 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpftc\" (UniqueName: \"kubernetes.io/projected/df4dd9e1-7bd1-43f3-abf6-c8e16405b291-kube-api-access-gpftc\") pod \"kuadrant-operator-controller-manager-6b6fb56846-pj2h4\" (UID: \"df4dd9e1-7bd1-43f3-abf6-c8e16405b291\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:29.463755 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.463731 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:29.551774 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.551742 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:29.789532 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.789447 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4"] Oct 29 23:22:29.794929 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:29.794898 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:29.804819 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:22:29.804795 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf4dd9e1_7bd1_43f3_abf6_c8e16405b291.slice/crio-bba17ba041d72fc8256ca94215f766eedf1daf915d0a7c69d6bb1118d3b51942 WatchSource:0}: Error finding container bba17ba041d72fc8256ca94215f766eedf1daf915d0a7c69d6bb1118d3b51942: Status 404 returned error can't find the container with id bba17ba041d72fc8256ca94215f766eedf1daf915d0a7c69d6bb1118d3b51942 Oct 29 23:22:30.303790 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:30.303759 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" event={"ID":"df4dd9e1-7bd1-43f3-abf6-c8e16405b291","Type":"ContainerStarted","Data":"bba17ba041d72fc8256ca94215f766eedf1daf915d0a7c69d6bb1118d3b51942"} Oct 29 23:22:30.906418 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:30.906341 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:31.119744 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:31.119707 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:31.766594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:31.766558 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d6dd76dc-pcg45"] Oct 29 23:22:31.778136 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:31.778107 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d6dd76dc-pcg45" Oct 29 23:22:31.795428 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:31.795391 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d6dd76dc-pcg45"] Oct 29 23:22:31.888422 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:31.888389 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-c24jf\"" Oct 29 23:22:31.924478 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:31.924436 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w282t\" (UniqueName: \"kubernetes.io/projected/632b1b15-84f4-4886-a72e-6890e4f0ac19-kube-api-access-w282t\") pod \"authorino-operator-68d6dd76dc-pcg45\" (UID: \"632b1b15-84f4-4886-a72e-6890e4f0ac19\") " pod="kuadrant-system/authorino-operator-68d6dd76dc-pcg45" Oct 29 23:22:32.025306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:32.025199 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w282t\" (UniqueName: \"kubernetes.io/projected/632b1b15-84f4-4886-a72e-6890e4f0ac19-kube-api-access-w282t\") pod \"authorino-operator-68d6dd76dc-pcg45\" (UID: \"632b1b15-84f4-4886-a72e-6890e4f0ac19\") " pod="kuadrant-system/authorino-operator-68d6dd76dc-pcg45" Oct 29 23:22:32.052159 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:32.052128 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w282t\" (UniqueName: \"kubernetes.io/projected/632b1b15-84f4-4886-a72e-6890e4f0ac19-kube-api-access-w282t\") pod \"authorino-operator-68d6dd76dc-pcg45\" (UID: \"632b1b15-84f4-4886-a72e-6890e4f0ac19\") " pod="kuadrant-system/authorino-operator-68d6dd76dc-pcg45" Oct 29 23:22:32.091967 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:32.091934 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d6dd76dc-pcg45" Oct 29 23:22:32.400644 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:32.400616 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d6dd76dc-pcg45"] Oct 29 23:22:32.403610 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:22:32.403578 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod632b1b15_84f4_4886_a72e_6890e4f0ac19.slice/crio-7ee2c1aeff15b4d73e63fa4be22c31f1c09148b5aad1cf23a669333dc7d8a9ad WatchSource:0}: Error finding container 7ee2c1aeff15b4d73e63fa4be22c31f1c09148b5aad1cf23a669333dc7d8a9ad: Status 404 returned error can't find the container with id 7ee2c1aeff15b4d73e63fa4be22c31f1c09148b5aad1cf23a669333dc7d8a9ad Oct 29 23:22:32.411135 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:32.411103 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:32.647249 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:32.647220 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:33.322903 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:33.322868 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d6dd76dc-pcg45" event={"ID":"632b1b15-84f4-4886-a72e-6890e4f0ac19","Type":"ContainerStarted","Data":"7ee2c1aeff15b4d73e63fa4be22c31f1c09148b5aad1cf23a669333dc7d8a9ad"} Oct 29 23:22:34.329091 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:34.328989 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" event={"ID":"df4dd9e1-7bd1-43f3-abf6-c8e16405b291","Type":"ContainerStarted","Data":"e4ad895c9b040673e200f2d91a7bda80192f5d9b943db194651fdb7ffe2efbb8"} Oct 29 23:22:34.329091 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:34.329058 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:34.331780 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:34.331756 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d6dd76dc-pcg45" event={"ID":"632b1b15-84f4-4886-a72e-6890e4f0ac19","Type":"ContainerStarted","Data":"babafb8eb5bde6ba5fbaf010d6418f5445dbcc1277c06d501724bd0617ef4a0a"} Oct 29 23:22:34.331907 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:34.331832 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d6dd76dc-pcg45" Oct 29 23:22:34.403232 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:34.403181 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" podStartSLOduration=1.208355931 podStartE2EDuration="5.403166089s" totalImagesPullingTime="4.194810158s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:22:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:22:34.389597898 +0000 UTC m=+581.756345535" watchObservedRunningTime="2026-10-29 23:22:34.403166089 +0000 UTC m=+581.769913777" Oct 29 23:22:34.439845 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:34.439784 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d6dd76dc-pcg45" podStartSLOduration=2.221323489 podStartE2EDuration="3.439769944s" totalImagesPullingTime="1.218446455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:22:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:22:34.438705003 +0000 UTC m=+581.805452634" watchObservedRunningTime="2026-10-29 23:22:34.439769944 +0000 UTC m=+581.806517574" Oct 29 23:22:34.662306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:34.662254 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:36.036905 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:36.036869 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:37.601253 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:37.601216 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:37.760391 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:37.760353 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:39.792774 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:39.792735 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:39.948409 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:39.948378 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:40.200069 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:40.200039 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:40.349340 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:40.349311 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:40.609694 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:40.609597 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:40.767890 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:40.767852 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:43.520623 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:43.520593 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:45.339409 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:45.339368 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d6dd76dc-pcg45" Oct 29 23:22:45.339804 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:45.339443 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:48.639121 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:48.639084 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:49.071865 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.071753 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4"] Oct 29 23:22:49.072070 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.072023 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" podUID="df4dd9e1-7bd1-43f3-abf6-c8e16405b291" containerName="manager" containerID="cri-o://e4ad895c9b040673e200f2d91a7bda80192f5d9b943db194651fdb7ffe2efbb8" gracePeriod=2 Oct 29 23:22:49.081297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.081253 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4"] Oct 29 23:22:49.090008 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:22:49.089979 5056 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-pj2h4\" is forbidden: User \"system:node:ip-10-0-130-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-86.ec2.internal' and this object" podUID="df4dd9e1-7bd1-43f3-abf6-c8e16405b291" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:49.116182 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.116155 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4"] Oct 29 23:22:49.116622 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.116608 5056 cpu_manager.go:414] "containerMap: removing container" podUID="df4dd9e1-7bd1-43f3-abf6-c8e16405b291" containerName="manager" Oct 29 23:22:49.116674 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.116623 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df4dd9e1-7bd1-43f3-abf6-c8e16405b291" containerName="manager" Oct 29 23:22:49.116712 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.116683 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="df4dd9e1-7bd1-43f3-abf6-c8e16405b291" containerName="manager" Oct 29 23:22:49.119698 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.119675 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" Oct 29 23:22:49.134329 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:22:49.134112 5056 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-pj2h4\" is forbidden: User \"system:node:ip-10-0-130-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-86.ec2.internal' and this object" podUID="df4dd9e1-7bd1-43f3-abf6-c8e16405b291" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:49.148081 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.148055 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4"] Oct 29 23:22:49.177639 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.177613 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73b41495-1708-475d-bd20-20d0fd13ca9a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-twrw4\" (UID: \"73b41495-1708-475d-bd20-20d0fd13ca9a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" Oct 29 23:22:49.177803 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.177673 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z28g7\" (UniqueName: \"kubernetes.io/projected/73b41495-1708-475d-bd20-20d0fd13ca9a-kube-api-access-z28g7\") pod \"kuadrant-operator-controller-manager-6b6fb56846-twrw4\" (UID: \"73b41495-1708-475d-bd20-20d0fd13ca9a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" Oct 29 23:22:49.278794 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.278758 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73b41495-1708-475d-bd20-20d0fd13ca9a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-twrw4\" (UID: \"73b41495-1708-475d-bd20-20d0fd13ca9a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" Oct 29 23:22:49.278928 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.278839 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z28g7\" (UniqueName: \"kubernetes.io/projected/73b41495-1708-475d-bd20-20d0fd13ca9a-kube-api-access-z28g7\") pod \"kuadrant-operator-controller-manager-6b6fb56846-twrw4\" (UID: \"73b41495-1708-475d-bd20-20d0fd13ca9a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" Oct 29 23:22:49.279132 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.279112 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73b41495-1708-475d-bd20-20d0fd13ca9a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-twrw4\" (UID: \"73b41495-1708-475d-bd20-20d0fd13ca9a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" Oct 29 23:22:49.313620 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.313589 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z28g7\" (UniqueName: \"kubernetes.io/projected/73b41495-1708-475d-bd20-20d0fd13ca9a-kube-api-access-z28g7\") pod \"kuadrant-operator-controller-manager-6b6fb56846-twrw4\" (UID: \"73b41495-1708-475d-bd20-20d0fd13ca9a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" Oct 29 23:22:49.333283 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.333223 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:49.348678 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:22:49.348652 5056 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-pj2h4\" is forbidden: User \"system:node:ip-10-0-130-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-86.ec2.internal' and this object" podUID="df4dd9e1-7bd1-43f3-abf6-c8e16405b291" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:49.379756 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.379733 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df4dd9e1-7bd1-43f3-abf6-c8e16405b291-kube-api-access-gpftc\" (UniqueName: \"kubernetes.io/projected/df4dd9e1-7bd1-43f3-abf6-c8e16405b291-kube-api-access-gpftc\") pod \"df4dd9e1-7bd1-43f3-abf6-c8e16405b291\" (UID: \"df4dd9e1-7bd1-43f3-abf6-c8e16405b291\") " Oct 29 23:22:49.379854 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.379773 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df4dd9e1-7bd1-43f3-abf6-c8e16405b291-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df4dd9e1-7bd1-43f3-abf6-c8e16405b291-extensions-socket-volume\") pod \"df4dd9e1-7bd1-43f3-abf6-c8e16405b291\" (UID: \"df4dd9e1-7bd1-43f3-abf6-c8e16405b291\") " Oct 29 23:22:49.380287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.380239 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df4dd9e1-7bd1-43f3-abf6-c8e16405b291-extensions-socket-volume" pod "df4dd9e1-7bd1-43f3-abf6-c8e16405b291" (UID: "df4dd9e1-7bd1-43f3-abf6-c8e16405b291"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:22:49.381826 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.381799 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df4dd9e1-7bd1-43f3-abf6-c8e16405b291-kube-api-access-gpftc" pod "df4dd9e1-7bd1-43f3-abf6-c8e16405b291" (UID: "df4dd9e1-7bd1-43f3-abf6-c8e16405b291"). InnerVolumeSpecName "kube-api-access-gpftc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:22:49.400655 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.400629 5056 generic.go:356] "Generic (PLEG): container finished" podID="df4dd9e1-7bd1-43f3-abf6-c8e16405b291" containerID="e4ad895c9b040673e200f2d91a7bda80192f5d9b943db194651fdb7ffe2efbb8" exitCode=0 Oct 29 23:22:49.400761 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.400671 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:49.400761 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.400694 5056 scope.go:122] "RemoveContainer" containerID="e4ad895c9b040673e200f2d91a7bda80192f5d9b943db194651fdb7ffe2efbb8" Oct 29 23:22:49.409552 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.409536 5056 scope.go:122] "RemoveContainer" containerID="e4ad895c9b040673e200f2d91a7bda80192f5d9b943db194651fdb7ffe2efbb8" Oct 29 23:22:49.409860 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:22:49.409841 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4ad895c9b040673e200f2d91a7bda80192f5d9b943db194651fdb7ffe2efbb8\": container with ID starting with e4ad895c9b040673e200f2d91a7bda80192f5d9b943db194651fdb7ffe2efbb8 not found: ID does not exist" containerID="e4ad895c9b040673e200f2d91a7bda80192f5d9b943db194651fdb7ffe2efbb8" Oct 29 23:22:49.409914 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.409868 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4ad895c9b040673e200f2d91a7bda80192f5d9b943db194651fdb7ffe2efbb8"} err="failed to get container status \"e4ad895c9b040673e200f2d91a7bda80192f5d9b943db194651fdb7ffe2efbb8\": rpc error: code = NotFound desc = could not find container \"e4ad895c9b040673e200f2d91a7bda80192f5d9b943db194651fdb7ffe2efbb8\": container with ID starting with e4ad895c9b040673e200f2d91a7bda80192f5d9b943db194651fdb7ffe2efbb8 not found: ID does not exist" Oct 29 23:22:49.416002 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:22:49.415978 5056 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-pj2h4\" is forbidden: User \"system:node:ip-10-0-130-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-86.ec2.internal' and this object" podUID="df4dd9e1-7bd1-43f3-abf6-c8e16405b291" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:49.430533 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:22:49.430506 5056 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-pj2h4\" is forbidden: User \"system:node:ip-10-0-130-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-86.ec2.internal' and this object" podUID="df4dd9e1-7bd1-43f3-abf6-c8e16405b291" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:49.471706 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.471684 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" Oct 29 23:22:49.480503 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.480484 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpftc\" (UniqueName: \"kubernetes.io/projected/df4dd9e1-7bd1-43f3-abf6-c8e16405b291-kube-api-access-gpftc\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:22:49.480592 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.480508 5056 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/df4dd9e1-7bd1-43f3-abf6-c8e16405b291-extensions-socket-volume\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:22:49.750072 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.750045 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:49.798627 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:49.798606 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4"] Oct 29 23:22:49.800941 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:22:49.800916 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73b41495_1708_475d_bd20_20d0fd13ca9a.slice/crio-433f317c8f51b537c2747e9d32b907c8d1bbd4d701b30dc5b575c7e1b5eecaa4 WatchSource:0}: Error finding container 433f317c8f51b537c2747e9d32b907c8d1bbd4d701b30dc5b575c7e1b5eecaa4: Status 404 returned error can't find the container with id 433f317c8f51b537c2747e9d32b907c8d1bbd4d701b30dc5b575c7e1b5eecaa4 Oct 29 23:22:50.406778 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:50.406742 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" event={"ID":"73b41495-1708-475d-bd20-20d0fd13ca9a","Type":"ContainerStarted","Data":"af1633083eb8fe4a4f80fddfad7ffbf4d93582cda8332486472a27a02f15e127"} Oct 29 23:22:50.406961 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:50.406885 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" Oct 29 23:22:50.406961 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:50.406908 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" event={"ID":"73b41495-1708-475d-bd20-20d0fd13ca9a","Type":"ContainerStarted","Data":"433f317c8f51b537c2747e9d32b907c8d1bbd4d701b30dc5b575c7e1b5eecaa4"} Oct 29 23:22:50.478031 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:50.477959 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" podStartSLOduration=1.477946104 podStartE2EDuration="1.477946104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:22:50.46481197 +0000 UTC m=+597.831559601" watchObservedRunningTime="2026-10-29 23:22:50.477946104 +0000 UTC m=+597.844693735" Oct 29 23:22:50.479306 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:22:50.479244 5056 status_manager.go:1045] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6b6fb56846-pj2h4\" is forbidden: User \"system:node:ip-10-0-130-86.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-86.ec2.internal' and this object" podUID="df4dd9e1-7bd1-43f3-abf6-c8e16405b291" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-pj2h4" Oct 29 23:22:50.769225 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:50.769149 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df4dd9e1-7bd1-43f3-abf6-c8e16405b291" path="/var/lib/kubelet/pods/df4dd9e1-7bd1-43f3-abf6-c8e16405b291/volumes" Oct 29 23:22:51.082310 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:51.082213 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:51.235530 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:51.235496 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:52.414241 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:22:52.414204 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:22:52.810776 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:52.810751 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:22:52.813075 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:52.813055 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:22:55.817613 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:55.817576 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:22:55.972568 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:22:55.972526 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:01.027763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:01.027731 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:01.178735 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:01.178701 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:01.414815 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:01.414787 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" Oct 29 23:23:01.442500 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:01.442469 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:01.640273 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:01.640240 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:01.792752 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:01.792673 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:03.755189 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:03.755154 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4"] Oct 29 23:23:03.755707 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:03.755371 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" podUID="73b41495-1708-475d-bd20-20d0fd13ca9a" containerName="manager" containerID="cri-o://af1633083eb8fe4a4f80fddfad7ffbf4d93582cda8332486472a27a02f15e127" gracePeriod=10 Oct 29 23:23:03.961941 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:03.961900 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq"] Oct 29 23:23:03.965140 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:03.965116 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" Oct 29 23:23:03.992174 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:03.992129 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq"] Oct 29 23:23:04.000493 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.000469 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crgr2\" (UniqueName: \"kubernetes.io/projected/ba820266-433a-4903-bfa6-113a385e5327-kube-api-access-crgr2\") pod \"kuadrant-operator-controller-manager-57bb9bfff-zkgpq\" (UID: \"ba820266-433a-4903-bfa6-113a385e5327\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" Oct 29 23:23:04.000616 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.000523 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba820266-433a-4903-bfa6-113a385e5327-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-zkgpq\" (UID: \"ba820266-433a-4903-bfa6-113a385e5327\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" Oct 29 23:23:04.013678 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.013621 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" Oct 29 23:23:04.101360 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.101327 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/73b41495-1708-475d-bd20-20d0fd13ca9a-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73b41495-1708-475d-bd20-20d0fd13ca9a-extensions-socket-volume\") pod \"73b41495-1708-475d-bd20-20d0fd13ca9a\" (UID: \"73b41495-1708-475d-bd20-20d0fd13ca9a\") " Oct 29 23:23:04.101505 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.101395 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73b41495-1708-475d-bd20-20d0fd13ca9a-kube-api-access-z28g7\" (UniqueName: \"kubernetes.io/projected/73b41495-1708-475d-bd20-20d0fd13ca9a-kube-api-access-z28g7\") pod \"73b41495-1708-475d-bd20-20d0fd13ca9a\" (UID: \"73b41495-1708-475d-bd20-20d0fd13ca9a\") " Oct 29 23:23:04.101569 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.101514 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crgr2\" (UniqueName: \"kubernetes.io/projected/ba820266-433a-4903-bfa6-113a385e5327-kube-api-access-crgr2\") pod \"kuadrant-operator-controller-manager-57bb9bfff-zkgpq\" (UID: \"ba820266-433a-4903-bfa6-113a385e5327\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" Oct 29 23:23:04.101606 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.101567 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba820266-433a-4903-bfa6-113a385e5327-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-zkgpq\" (UID: \"ba820266-433a-4903-bfa6-113a385e5327\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" Oct 29 23:23:04.101748 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.101722 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/73b41495-1708-475d-bd20-20d0fd13ca9a-extensions-socket-volume" pod "73b41495-1708-475d-bd20-20d0fd13ca9a" (UID: "73b41495-1708-475d-bd20-20d0fd13ca9a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:23:04.101891 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.101873 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba820266-433a-4903-bfa6-113a385e5327-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-zkgpq\" (UID: \"ba820266-433a-4903-bfa6-113a385e5327\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" Oct 29 23:23:04.103462 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.103440 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73b41495-1708-475d-bd20-20d0fd13ca9a-kube-api-access-z28g7" pod "73b41495-1708-475d-bd20-20d0fd13ca9a" (UID: "73b41495-1708-475d-bd20-20d0fd13ca9a"). InnerVolumeSpecName "kube-api-access-z28g7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:23:04.126579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.126547 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crgr2\" (UniqueName: \"kubernetes.io/projected/ba820266-433a-4903-bfa6-113a385e5327-kube-api-access-crgr2\") pod \"kuadrant-operator-controller-manager-57bb9bfff-zkgpq\" (UID: \"ba820266-433a-4903-bfa6-113a385e5327\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" Oct 29 23:23:04.202004 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.201967 5056 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/73b41495-1708-475d-bd20-20d0fd13ca9a-extensions-socket-volume\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:23:04.202004 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.201996 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z28g7\" (UniqueName: \"kubernetes.io/projected/73b41495-1708-475d-bd20-20d0fd13ca9a-kube-api-access-z28g7\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:23:04.281135 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.281047 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" Oct 29 23:23:04.470014 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.469982 5056 generic.go:356] "Generic (PLEG): container finished" podID="73b41495-1708-475d-bd20-20d0fd13ca9a" containerID="af1633083eb8fe4a4f80fddfad7ffbf4d93582cda8332486472a27a02f15e127" exitCode=0 Oct 29 23:23:04.470211 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.470034 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" Oct 29 23:23:04.470211 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.470067 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" event={"ID":"73b41495-1708-475d-bd20-20d0fd13ca9a","Type":"ContainerDied","Data":"af1633083eb8fe4a4f80fddfad7ffbf4d93582cda8332486472a27a02f15e127"} Oct 29 23:23:04.470211 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.470121 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4" event={"ID":"73b41495-1708-475d-bd20-20d0fd13ca9a","Type":"ContainerDied","Data":"433f317c8f51b537c2747e9d32b907c8d1bbd4d701b30dc5b575c7e1b5eecaa4"} Oct 29 23:23:04.470211 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.470145 5056 scope.go:122] "RemoveContainer" containerID="af1633083eb8fe4a4f80fddfad7ffbf4d93582cda8332486472a27a02f15e127" Oct 29 23:23:04.478828 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.478811 5056 scope.go:122] "RemoveContainer" containerID="af1633083eb8fe4a4f80fddfad7ffbf4d93582cda8332486472a27a02f15e127" Oct 29 23:23:04.479101 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:23:04.479084 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af1633083eb8fe4a4f80fddfad7ffbf4d93582cda8332486472a27a02f15e127\": container with ID starting with af1633083eb8fe4a4f80fddfad7ffbf4d93582cda8332486472a27a02f15e127 not found: ID does not exist" containerID="af1633083eb8fe4a4f80fddfad7ffbf4d93582cda8332486472a27a02f15e127" Oct 29 23:23:04.479150 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.479110 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af1633083eb8fe4a4f80fddfad7ffbf4d93582cda8332486472a27a02f15e127"} err="failed to get container status \"af1633083eb8fe4a4f80fddfad7ffbf4d93582cda8332486472a27a02f15e127\": rpc error: code = NotFound desc = could not find container \"af1633083eb8fe4a4f80fddfad7ffbf4d93582cda8332486472a27a02f15e127\": container with ID starting with af1633083eb8fe4a4f80fddfad7ffbf4d93582cda8332486472a27a02f15e127 not found: ID does not exist" Oct 29 23:23:04.525121 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.525095 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4"] Oct 29 23:23:04.538356 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.538300 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-twrw4"] Oct 29 23:23:04.597703 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.597672 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq"] Oct 29 23:23:04.600058 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:23:04.600030 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba820266_433a_4903_bfa6_113a385e5327.slice/crio-b5870afbb92dabcbbd5d5188ceb76f96c601602d49c4d7b91aac29b09c4e18ac WatchSource:0}: Error finding container b5870afbb92dabcbbd5d5188ceb76f96c601602d49c4d7b91aac29b09c4e18ac: Status 404 returned error can't find the container with id b5870afbb92dabcbbd5d5188ceb76f96c601602d49c4d7b91aac29b09c4e18ac Oct 29 23:23:04.769797 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:04.769761 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73b41495-1708-475d-bd20-20d0fd13ca9a" path="/var/lib/kubelet/pods/73b41495-1708-475d-bd20-20d0fd13ca9a/volumes" Oct 29 23:23:05.476412 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:05.476374 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" event={"ID":"ba820266-433a-4903-bfa6-113a385e5327","Type":"ContainerStarted","Data":"cb91254fdd483400d84cf3e5c6cf5d8c4de0ec52eab7659543ebb60723d4b8c8"} Oct 29 23:23:05.476596 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:05.476430 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" event={"ID":"ba820266-433a-4903-bfa6-113a385e5327","Type":"ContainerStarted","Data":"b5870afbb92dabcbbd5d5188ceb76f96c601602d49c4d7b91aac29b09c4e18ac"} Oct 29 23:23:05.476596 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:05.476522 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" Oct 29 23:23:05.559717 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:05.559669 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" podStartSLOduration=2.559655837 podStartE2EDuration="2.559655837s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:23:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:23:05.54569195 +0000 UTC m=+612.912439580" watchObservedRunningTime="2026-10-29 23:23:05.559655837 +0000 UTC m=+612.926403467" Oct 29 23:23:10.045658 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:10.045614 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:10.204418 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:10.204382 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:12.497531 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:12.497494 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:12.649705 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:12.649666 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:16.484762 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:16.484732 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" Oct 29 23:23:17.256334 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:17.256244 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:17.433090 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:17.433056 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:20.969432 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:20.969399 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:21.122030 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:21.121997 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:22.415038 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:23:22.415002 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:23:29.835196 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:29.835165 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:29.985363 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:29.985331 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:34.550279 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.550219 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-ks9kp"] Oct 29 23:23:34.550667 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.550614 5056 cpu_manager.go:414] "containerMap: removing container" podUID="73b41495-1708-475d-bd20-20d0fd13ca9a" containerName="manager" Oct 29 23:23:34.550667 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.550627 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73b41495-1708-475d-bd20-20d0fd13ca9a" containerName="manager" Oct 29 23:23:34.550736 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.550716 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="73b41495-1708-475d-bd20-20d0fd13ca9a" containerName="manager" Oct 29 23:23:34.553925 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.553906 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" Oct 29 23:23:34.573629 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.573597 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-tbfkq\"" Oct 29 23:23:34.573760 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.573666 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Oct 29 23:23:34.584552 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.584528 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-ks9kp"] Oct 29 23:23:34.585970 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.585946 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:34.657557 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.657520 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/35a09898-2d99-4a5d-b2ec-666e80a2fe68-config-file\") pod \"limitador-limitador-74dfcfb894-ks9kp\" (UID: \"35a09898-2d99-4a5d-b2ec-666e80a2fe68\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" Oct 29 23:23:34.657749 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.657652 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvzh2\" (UniqueName: \"kubernetes.io/projected/35a09898-2d99-4a5d-b2ec-666e80a2fe68-kube-api-access-jvzh2\") pod \"limitador-limitador-74dfcfb894-ks9kp\" (UID: \"35a09898-2d99-4a5d-b2ec-666e80a2fe68\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" Oct 29 23:23:34.685548 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.685518 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-ks9kp"] Oct 29 23:23:34.745771 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.745740 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:34.759041 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.759015 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/35a09898-2d99-4a5d-b2ec-666e80a2fe68-config-file\") pod \"limitador-limitador-74dfcfb894-ks9kp\" (UID: \"35a09898-2d99-4a5d-b2ec-666e80a2fe68\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" Oct 29 23:23:34.759169 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.759068 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvzh2\" (UniqueName: \"kubernetes.io/projected/35a09898-2d99-4a5d-b2ec-666e80a2fe68-kube-api-access-jvzh2\") pod \"limitador-limitador-74dfcfb894-ks9kp\" (UID: \"35a09898-2d99-4a5d-b2ec-666e80a2fe68\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" Oct 29 23:23:34.759793 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.759761 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/35a09898-2d99-4a5d-b2ec-666e80a2fe68-config-file\") pod \"limitador-limitador-74dfcfb894-ks9kp\" (UID: \"35a09898-2d99-4a5d-b2ec-666e80a2fe68\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" Oct 29 23:23:34.783723 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.783700 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvzh2\" (UniqueName: \"kubernetes.io/projected/35a09898-2d99-4a5d-b2ec-666e80a2fe68-kube-api-access-jvzh2\") pod \"limitador-limitador-74dfcfb894-ks9kp\" (UID: \"35a09898-2d99-4a5d-b2ec-666e80a2fe68\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" Oct 29 23:23:34.866885 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.866812 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" Oct 29 23:23:34.903399 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:34.903371 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:35.166120 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:35.166092 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-ks9kp"] Oct 29 23:23:35.168926 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:23:35.168897 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35a09898_2d99_4a5d_b2ec_666e80a2fe68.slice/crio-678c2395fe2e9ed4038db2433849a231fd269186913760c8729cedc144bb0d03 WatchSource:0}: Error finding container 678c2395fe2e9ed4038db2433849a231fd269186913760c8729cedc144bb0d03: Status 404 returned error can't find the container with id 678c2395fe2e9ed4038db2433849a231fd269186913760c8729cedc144bb0d03 Oct 29 23:23:35.204481 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:35.204453 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:35.372699 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:35.372668 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:35.539536 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:35.539457 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:35.619329 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:35.619298 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" event={"ID":"35a09898-2d99-4a5d-b2ec-666e80a2fe68","Type":"ContainerStarted","Data":"678c2395fe2e9ed4038db2433849a231fd269186913760c8729cedc144bb0d03"} Oct 29 23:23:35.747619 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:35.747585 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:36.624970 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:36.624930 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" event={"ID":"35a09898-2d99-4a5d-b2ec-666e80a2fe68","Type":"ContainerStarted","Data":"d511a9c5be7fc61239e662aa963a999b832287d33bd91f3adb08ba252621c0a5"} Oct 29 23:23:36.625439 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:36.625017 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" Oct 29 23:23:36.671518 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:36.671471 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" podStartSLOduration=1.373542247 podStartE2EDuration="2.671457315s" totalImagesPullingTime="1.297915068s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:23:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:23:36.670863044 +0000 UTC m=+644.037610675" watchObservedRunningTime="2026-10-29 23:23:36.671457315 +0000 UTC m=+644.038204946" Oct 29 23:23:39.793185 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:39.793153 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:43.650064 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:43.650018 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:47.629840 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:47.629808 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" Oct 29 23:23:48.962196 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:48.962164 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-ks9kp"] Oct 29 23:23:48.962649 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:48.962360 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" podUID="35a09898-2d99-4a5d-b2ec-666e80a2fe68" containerName="limitador" containerID="cri-o://d511a9c5be7fc61239e662aa963a999b832287d33bd91f3adb08ba252621c0a5" gracePeriod=30 Oct 29 23:23:49.503692 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:49.503668 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" Oct 29 23:23:49.580957 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:49.580888 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/35a09898-2d99-4a5d-b2ec-666e80a2fe68-config-file\" (UniqueName: \"kubernetes.io/configmap/35a09898-2d99-4a5d-b2ec-666e80a2fe68-config-file\") pod \"35a09898-2d99-4a5d-b2ec-666e80a2fe68\" (UID: \"35a09898-2d99-4a5d-b2ec-666e80a2fe68\") " Oct 29 23:23:49.580957 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:49.580923 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35a09898-2d99-4a5d-b2ec-666e80a2fe68-kube-api-access-jvzh2\" (UniqueName: \"kubernetes.io/projected/35a09898-2d99-4a5d-b2ec-666e80a2fe68-kube-api-access-jvzh2\") pod \"35a09898-2d99-4a5d-b2ec-666e80a2fe68\" (UID: \"35a09898-2d99-4a5d-b2ec-666e80a2fe68\") " Oct 29 23:23:49.581342 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:49.581314 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35a09898-2d99-4a5d-b2ec-666e80a2fe68-config-file" pod "35a09898-2d99-4a5d-b2ec-666e80a2fe68" (UID: "35a09898-2d99-4a5d-b2ec-666e80a2fe68"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:23:49.583037 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:49.583012 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35a09898-2d99-4a5d-b2ec-666e80a2fe68-kube-api-access-jvzh2" pod "35a09898-2d99-4a5d-b2ec-666e80a2fe68" (UID: "35a09898-2d99-4a5d-b2ec-666e80a2fe68"). InnerVolumeSpecName "kube-api-access-jvzh2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:23:49.680834 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:49.680804 5056 generic.go:356] "Generic (PLEG): container finished" podID="35a09898-2d99-4a5d-b2ec-666e80a2fe68" containerID="d511a9c5be7fc61239e662aa963a999b832287d33bd91f3adb08ba252621c0a5" exitCode=0 Oct 29 23:23:49.680994 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:49.680848 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" Oct 29 23:23:49.680994 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:49.680891 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" event={"ID":"35a09898-2d99-4a5d-b2ec-666e80a2fe68","Type":"ContainerDied","Data":"d511a9c5be7fc61239e662aa963a999b832287d33bd91f3adb08ba252621c0a5"} Oct 29 23:23:49.680994 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:49.680942 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-ks9kp" event={"ID":"35a09898-2d99-4a5d-b2ec-666e80a2fe68","Type":"ContainerDied","Data":"678c2395fe2e9ed4038db2433849a231fd269186913760c8729cedc144bb0d03"} Oct 29 23:23:49.680994 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:49.680970 5056 scope.go:122] "RemoveContainer" containerID="d511a9c5be7fc61239e662aa963a999b832287d33bd91f3adb08ba252621c0a5" Oct 29 23:23:49.681574 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:49.681552 5056 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/35a09898-2d99-4a5d-b2ec-666e80a2fe68-config-file\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:23:49.681699 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:49.681578 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvzh2\" (UniqueName: \"kubernetes.io/projected/35a09898-2d99-4a5d-b2ec-666e80a2fe68-kube-api-access-jvzh2\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:23:49.690288 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:49.690271 5056 scope.go:122] "RemoveContainer" containerID="d511a9c5be7fc61239e662aa963a999b832287d33bd91f3adb08ba252621c0a5" Oct 29 23:23:49.690525 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:23:49.690509 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d511a9c5be7fc61239e662aa963a999b832287d33bd91f3adb08ba252621c0a5\": container with ID starting with d511a9c5be7fc61239e662aa963a999b832287d33bd91f3adb08ba252621c0a5 not found: ID does not exist" containerID="d511a9c5be7fc61239e662aa963a999b832287d33bd91f3adb08ba252621c0a5" Oct 29 23:23:49.690592 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:49.690533 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d511a9c5be7fc61239e662aa963a999b832287d33bd91f3adb08ba252621c0a5"} err="failed to get container status \"d511a9c5be7fc61239e662aa963a999b832287d33bd91f3adb08ba252621c0a5\": rpc error: code = NotFound desc = could not find container \"d511a9c5be7fc61239e662aa963a999b832287d33bd91f3adb08ba252621c0a5\": container with ID starting with d511a9c5be7fc61239e662aa963a999b832287d33bd91f3adb08ba252621c0a5 not found: ID does not exist" Oct 29 23:23:49.740791 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:49.740766 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-ks9kp"] Oct 29 23:23:49.748975 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:49.748954 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-ks9kp"] Oct 29 23:23:50.768327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:50.768292 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35a09898-2d99-4a5d-b2ec-666e80a2fe68" path="/var/lib/kubelet/pods/35a09898-2d99-4a5d-b2ec-666e80a2fe68/volumes" Oct 29 23:23:50.833042 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:50.832985 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:52.414388 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:23:52.414352 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:23:55.274976 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:55.274941 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:55.435937 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:55.435904 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:55.832195 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:55.832164 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-xvxk7"] Oct 29 23:23:55.832530 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:55.832517 5056 cpu_manager.go:414] "containerMap: removing container" podUID="35a09898-2d99-4a5d-b2ec-666e80a2fe68" containerName="limitador" Oct 29 23:23:55.832577 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:55.832531 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35a09898-2d99-4a5d-b2ec-666e80a2fe68" containerName="limitador" Oct 29 23:23:55.832611 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:55.832606 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="35a09898-2d99-4a5d-b2ec-666e80a2fe68" containerName="limitador" Oct 29 23:23:55.837034 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:55.837015 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xvxk7" Oct 29 23:23:55.857288 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:55.857246 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-xvxk7"] Oct 29 23:23:55.868312 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:55.868291 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 29 23:23:55.868433 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:55.868342 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 29 23:23:55.868433 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:55.868367 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Oct 29 23:23:55.868433 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:55.868422 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-rmhwj\"" Oct 29 23:23:55.935665 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:55.935628 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/067da039-3d5e-40f2-b2bf-a7ec57feed06-data\") pod \"postgres-7cdbf6dc44-xvxk7\" (UID: \"067da039-3d5e-40f2-b2bf-a7ec57feed06\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xvxk7" Oct 29 23:23:55.935822 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:55.935751 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m9sk\" (UniqueName: \"kubernetes.io/projected/067da039-3d5e-40f2-b2bf-a7ec57feed06-kube-api-access-9m9sk\") pod \"postgres-7cdbf6dc44-xvxk7\" (UID: \"067da039-3d5e-40f2-b2bf-a7ec57feed06\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xvxk7" Oct 29 23:23:55.960008 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:55.959960 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:23:56.036380 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:56.036353 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/067da039-3d5e-40f2-b2bf-a7ec57feed06-data\") pod \"postgres-7cdbf6dc44-xvxk7\" (UID: \"067da039-3d5e-40f2-b2bf-a7ec57feed06\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xvxk7" Oct 29 23:23:56.036506 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:56.036430 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m9sk\" (UniqueName: \"kubernetes.io/projected/067da039-3d5e-40f2-b2bf-a7ec57feed06-kube-api-access-9m9sk\") pod \"postgres-7cdbf6dc44-xvxk7\" (UID: \"067da039-3d5e-40f2-b2bf-a7ec57feed06\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xvxk7" Oct 29 23:23:56.036735 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:56.036717 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/067da039-3d5e-40f2-b2bf-a7ec57feed06-data\") pod \"postgres-7cdbf6dc44-xvxk7\" (UID: \"067da039-3d5e-40f2-b2bf-a7ec57feed06\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xvxk7" Oct 29 23:23:56.060788 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:56.060768 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m9sk\" (UniqueName: \"kubernetes.io/projected/067da039-3d5e-40f2-b2bf-a7ec57feed06-kube-api-access-9m9sk\") pod \"postgres-7cdbf6dc44-xvxk7\" (UID: \"067da039-3d5e-40f2-b2bf-a7ec57feed06\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xvxk7" Oct 29 23:23:56.152515 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:56.152492 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xvxk7" Oct 29 23:23:56.448917 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:56.448891 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-xvxk7"] Oct 29 23:23:56.450891 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:23:56.450860 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod067da039_3d5e_40f2_b2bf_a7ec57feed06.slice/crio-0bd98e80588f7da504b544745d21697b8333488e539e6749d1fc848cc16597df WatchSource:0}: Error finding container 0bd98e80588f7da504b544745d21697b8333488e539e6749d1fc848cc16597df: Status 404 returned error can't find the container with id 0bd98e80588f7da504b544745d21697b8333488e539e6749d1fc848cc16597df Oct 29 23:23:56.716453 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:23:56.716373 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xvxk7" event={"ID":"067da039-3d5e-40f2-b2bf-a7ec57feed06","Type":"ContainerStarted","Data":"0bd98e80588f7da504b544745d21697b8333488e539e6749d1fc848cc16597df"} Oct 29 23:24:01.743895 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:01.743850 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xvxk7" event={"ID":"067da039-3d5e-40f2-b2bf-a7ec57feed06","Type":"ContainerStarted","Data":"3b21319fc287c84c40c50a40fda686ef78e57b7ee09299d5381a263bf51b2f48"} Oct 29 23:24:01.744363 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:01.743951 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xvxk7" Oct 29 23:24:01.787028 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:01.786976 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xvxk7" podStartSLOduration=2.066950849 podStartE2EDuration="6.786963269s" totalImagesPullingTime="4.72001242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:23:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:24:01.786251469 +0000 UTC m=+669.152999096" watchObservedRunningTime="2026-10-29 23:24:01.786963269 +0000 UTC m=+669.153710900" Oct 29 23:24:07.777351 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:07.777283 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-xvxk7" Oct 29 23:24:08.277137 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:08.277106 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:24:08.441070 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:08.441032 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:24:17.493023 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.492989 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:24:17.566992 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.566959 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-577f596567-mss5z"] Oct 29 23:24:17.585774 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.585741 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-mss5z" Oct 29 23:24:17.594732 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.594704 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577f596567-mss5z"] Oct 29 23:24:17.606235 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.606209 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Oct 29 23:24:17.606389 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.606286 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5mgl4\"" Oct 29 23:24:17.614363 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.614344 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Oct 29 23:24:17.614473 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.614399 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Oct 29 23:24:17.653657 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.653625 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:24:17.725468 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.725434 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be79ca34-64ac-4752-bba8-be72842d52d8-metrics-certs\") pod \"maas-controller-577f596567-mss5z\" (UID: \"be79ca34-64ac-4752-bba8-be72842d52d8\") " pod="opendatahub/maas-controller-577f596567-mss5z" Oct 29 23:24:17.725622 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.725487 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be79ca34-64ac-4752-bba8-be72842d52d8-webhook-cert\") pod \"maas-controller-577f596567-mss5z\" (UID: \"be79ca34-64ac-4752-bba8-be72842d52d8\") " pod="opendatahub/maas-controller-577f596567-mss5z" Oct 29 23:24:17.725622 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.725607 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdfp2\" (UniqueName: \"kubernetes.io/projected/be79ca34-64ac-4752-bba8-be72842d52d8-kube-api-access-qdfp2\") pod \"maas-controller-577f596567-mss5z\" (UID: \"be79ca34-64ac-4752-bba8-be72842d52d8\") " pod="opendatahub/maas-controller-577f596567-mss5z" Oct 29 23:24:17.810397 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.810291 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:24:17.826449 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.826421 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdfp2\" (UniqueName: \"kubernetes.io/projected/be79ca34-64ac-4752-bba8-be72842d52d8-kube-api-access-qdfp2\") pod \"maas-controller-577f596567-mss5z\" (UID: \"be79ca34-64ac-4752-bba8-be72842d52d8\") " pod="opendatahub/maas-controller-577f596567-mss5z" Oct 29 23:24:17.826599 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.826475 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be79ca34-64ac-4752-bba8-be72842d52d8-metrics-certs\") pod \"maas-controller-577f596567-mss5z\" (UID: \"be79ca34-64ac-4752-bba8-be72842d52d8\") " pod="opendatahub/maas-controller-577f596567-mss5z" Oct 29 23:24:17.826599 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.826508 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be79ca34-64ac-4752-bba8-be72842d52d8-webhook-cert\") pod \"maas-controller-577f596567-mss5z\" (UID: \"be79ca34-64ac-4752-bba8-be72842d52d8\") " pod="opendatahub/maas-controller-577f596567-mss5z" Oct 29 23:24:17.828898 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.828879 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be79ca34-64ac-4752-bba8-be72842d52d8-webhook-cert\") pod \"maas-controller-577f596567-mss5z\" (UID: \"be79ca34-64ac-4752-bba8-be72842d52d8\") " pod="opendatahub/maas-controller-577f596567-mss5z" Oct 29 23:24:17.829008 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.828990 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be79ca34-64ac-4752-bba8-be72842d52d8-metrics-certs\") pod \"maas-controller-577f596567-mss5z\" (UID: \"be79ca34-64ac-4752-bba8-be72842d52d8\") " pod="opendatahub/maas-controller-577f596567-mss5z" Oct 29 23:24:17.851550 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.851530 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdfp2\" (UniqueName: \"kubernetes.io/projected/be79ca34-64ac-4752-bba8-be72842d52d8-kube-api-access-qdfp2\") pod \"maas-controller-577f596567-mss5z\" (UID: \"be79ca34-64ac-4752-bba8-be72842d52d8\") " pod="opendatahub/maas-controller-577f596567-mss5z" Oct 29 23:24:17.901025 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.900997 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-mss5z" Oct 29 23:24:17.920245 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.920220 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-95ff59779-z9sbx"] Oct 29 23:24:17.924734 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.924712 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-95ff59779-z9sbx" Oct 29 23:24:17.946537 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.946511 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-95ff59779-z9sbx"] Oct 29 23:24:17.962436 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:17.962079 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:24:18.028678 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:18.028647 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkctc\" (UniqueName: \"kubernetes.io/projected/4c8789da-b68f-46e9-b3e1-b380fd67a92b-kube-api-access-mkctc\") pod \"maas-controller-95ff59779-z9sbx\" (UID: \"4c8789da-b68f-46e9-b3e1-b380fd67a92b\") " pod="opendatahub/maas-controller-95ff59779-z9sbx" Oct 29 23:24:18.028832 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:18.028726 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c8789da-b68f-46e9-b3e1-b380fd67a92b-webhook-cert\") pod \"maas-controller-95ff59779-z9sbx\" (UID: \"4c8789da-b68f-46e9-b3e1-b380fd67a92b\") " pod="opendatahub/maas-controller-95ff59779-z9sbx" Oct 29 23:24:18.028832 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:18.028817 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c8789da-b68f-46e9-b3e1-b380fd67a92b-metrics-certs\") pod \"maas-controller-95ff59779-z9sbx\" (UID: \"4c8789da-b68f-46e9-b3e1-b380fd67a92b\") " pod="opendatahub/maas-controller-95ff59779-z9sbx" Oct 29 23:24:18.129868 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:18.129786 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c8789da-b68f-46e9-b3e1-b380fd67a92b-metrics-certs\") pod \"maas-controller-95ff59779-z9sbx\" (UID: \"4c8789da-b68f-46e9-b3e1-b380fd67a92b\") " pod="opendatahub/maas-controller-95ff59779-z9sbx" Oct 29 23:24:18.129868 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:18.129823 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkctc\" (UniqueName: \"kubernetes.io/projected/4c8789da-b68f-46e9-b3e1-b380fd67a92b-kube-api-access-mkctc\") pod \"maas-controller-95ff59779-z9sbx\" (UID: \"4c8789da-b68f-46e9-b3e1-b380fd67a92b\") " pod="opendatahub/maas-controller-95ff59779-z9sbx" Oct 29 23:24:18.130044 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:18.129877 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c8789da-b68f-46e9-b3e1-b380fd67a92b-webhook-cert\") pod \"maas-controller-95ff59779-z9sbx\" (UID: \"4c8789da-b68f-46e9-b3e1-b380fd67a92b\") " pod="opendatahub/maas-controller-95ff59779-z9sbx" Oct 29 23:24:18.132248 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:18.132224 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c8789da-b68f-46e9-b3e1-b380fd67a92b-metrics-certs\") pod \"maas-controller-95ff59779-z9sbx\" (UID: \"4c8789da-b68f-46e9-b3e1-b380fd67a92b\") " pod="opendatahub/maas-controller-95ff59779-z9sbx" Oct 29 23:24:18.132363 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:18.132249 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c8789da-b68f-46e9-b3e1-b380fd67a92b-webhook-cert\") pod \"maas-controller-95ff59779-z9sbx\" (UID: \"4c8789da-b68f-46e9-b3e1-b380fd67a92b\") " pod="opendatahub/maas-controller-95ff59779-z9sbx" Oct 29 23:24:18.153883 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:18.153848 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkctc\" (UniqueName: \"kubernetes.io/projected/4c8789da-b68f-46e9-b3e1-b380fd67a92b-kube-api-access-mkctc\") pod \"maas-controller-95ff59779-z9sbx\" (UID: \"4c8789da-b68f-46e9-b3e1-b380fd67a92b\") " pod="opendatahub/maas-controller-95ff59779-z9sbx" Oct 29 23:24:18.206642 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:18.206616 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577f596567-mss5z"] Oct 29 23:24:18.208554 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:24:18.208528 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe79ca34_64ac_4752_bba8_be72842d52d8.slice/crio-bcfeec58bceb29d0efe9978bb053954b14b442ee6596e93ad21926283058b9e0 WatchSource:0}: Error finding container bcfeec58bceb29d0efe9978bb053954b14b442ee6596e93ad21926283058b9e0: Status 404 returned error can't find the container with id bcfeec58bceb29d0efe9978bb053954b14b442ee6596e93ad21926283058b9e0 Oct 29 23:24:18.238884 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:18.238857 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-95ff59779-z9sbx" Oct 29 23:24:18.537536 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:18.537493 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-95ff59779-z9sbx"] Oct 29 23:24:18.539827 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:24:18.539784 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c8789da_b68f_46e9_b3e1_b380fd67a92b.slice/crio-e4daa3b886302fcfd9b12791aeef83b5247d9c8314accfba988d6790aa40f6c1 WatchSource:0}: Error finding container e4daa3b886302fcfd9b12791aeef83b5247d9c8314accfba988d6790aa40f6c1: Status 404 returned error can't find the container with id e4daa3b886302fcfd9b12791aeef83b5247d9c8314accfba988d6790aa40f6c1 Oct 29 23:24:18.821103 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:18.820984 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-mss5z" event={"ID":"be79ca34-64ac-4752-bba8-be72842d52d8","Type":"ContainerStarted","Data":"bcfeec58bceb29d0efe9978bb053954b14b442ee6596e93ad21926283058b9e0"} Oct 29 23:24:18.822687 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:18.822636 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-95ff59779-z9sbx" event={"ID":"4c8789da-b68f-46e9-b3e1-b380fd67a92b","Type":"ContainerStarted","Data":"e4daa3b886302fcfd9b12791aeef83b5247d9c8314accfba988d6790aa40f6c1"} Oct 29 23:24:21.283353 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:21.283298 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:24:21.538098 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:21.538013 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:24:21.839186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:21.839091 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-mss5z" event={"ID":"be79ca34-64ac-4752-bba8-be72842d52d8","Type":"ContainerStarted","Data":"f9a84b645b195ca4782cf6e48d9b74b70a1c4874ba3917603d7a1db052eccb52"} Oct 29 23:24:21.839186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:21.839152 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-577f596567-mss5z" Oct 29 23:24:21.840624 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:21.840595 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-95ff59779-z9sbx" event={"ID":"4c8789da-b68f-46e9-b3e1-b380fd67a92b","Type":"ContainerStarted","Data":"96f6594ad4178e3447f89a7e8427bea6d7ef3a2db4d7af2c369284fa9ebd6d7e"} Oct 29 23:24:21.840745 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:21.840652 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-95ff59779-z9sbx" Oct 29 23:24:21.884657 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:21.884608 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-577f596567-mss5z" podStartSLOduration=2.145229563 podStartE2EDuration="4.884595716s" totalImagesPullingTime="2.739366153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:24:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:24:21.884305304 +0000 UTC m=+689.251052934" watchObservedRunningTime="2026-10-29 23:24:21.884595716 +0000 UTC m=+689.251343364" Oct 29 23:24:21.934797 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:21.934747 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-95ff59779-z9sbx" podStartSLOduration=2.275293949 podStartE2EDuration="4.934732056s" totalImagesPullingTime="2.659438107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:24:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:24:21.934089566 +0000 UTC m=+689.300837180" watchObservedRunningTime="2026-10-29 23:24:21.934732056 +0000 UTC m=+689.301479728" Oct 29 23:24:22.417341 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:24:22.417304 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:24:24.670984 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.670949 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt"] Oct 29 23:24:24.674619 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.674602 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" Oct 29 23:24:24.691159 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.691135 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-88tvt\"" Oct 29 23:24:24.697441 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.697412 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\"" Oct 29 23:24:24.704311 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.704289 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\"" Oct 29 23:24:24.704648 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.704345 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\"" Oct 29 23:24:24.706464 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.706440 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt"] Oct 29 23:24:24.795652 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.795611 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-phwkt\" (UID: \"7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" Oct 29 23:24:24.795867 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.795668 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqxn4\" (UniqueName: \"kubernetes.io/projected/7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23-kube-api-access-mqxn4\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-phwkt\" (UID: \"7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" Oct 29 23:24:24.795867 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.795708 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-phwkt\" (UID: \"7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" Oct 29 23:24:24.795867 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.795769 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-phwkt\" (UID: \"7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" Oct 29 23:24:24.797952 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.797927 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:24:24.896663 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.896633 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-phwkt\" (UID: \"7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" Oct 29 23:24:24.896866 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.896686 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-phwkt\" (UID: \"7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" Oct 29 23:24:24.897017 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.896987 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqxn4\" (UniqueName: \"kubernetes.io/projected/7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23-kube-api-access-mqxn4\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-phwkt\" (UID: \"7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" Oct 29 23:24:24.897093 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.897070 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-phwkt\" (UID: \"7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" Oct 29 23:24:24.897635 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.897612 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-phwkt\" (UID: \"7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" Oct 29 23:24:24.897762 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.897668 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-phwkt\" (UID: \"7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" Oct 29 23:24:24.899311 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.899291 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-phwkt\" (UID: \"7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" Oct 29 23:24:24.921146 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.921087 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqxn4\" (UniqueName: \"kubernetes.io/projected/7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23-kube-api-access-mqxn4\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-phwkt\" (UID: \"7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" Oct 29 23:24:24.979218 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.979163 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:24:24.988142 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:24.988120 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" Oct 29 23:24:25.286522 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:25.286498 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt"] Oct 29 23:24:25.288872 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:24:25.288844 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7da3a0b7_1f7f_4b3e_a8f5_cf031895ef23.slice/crio-04e8a04375ded033dbde753a5a6d1a66580a570e7d40e9750ab895c56196f642 WatchSource:0}: Error finding container 04e8a04375ded033dbde753a5a6d1a66580a570e7d40e9750ab895c56196f642: Status 404 returned error can't find the container with id 04e8a04375ded033dbde753a5a6d1a66580a570e7d40e9750ab895c56196f642 Oct 29 23:24:25.861655 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:25.861613 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" event={"ID":"7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23","Type":"ContainerStarted","Data":"04e8a04375ded033dbde753a5a6d1a66580a570e7d40e9750ab895c56196f642"} Oct 29 23:24:30.659582 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:30.659541 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:24:30.809415 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:30.809373 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:24:32.847960 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:32.847920 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-95ff59779-z9sbx" Oct 29 23:24:32.848488 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:32.848011 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-577f596567-mss5z" Oct 29 23:24:32.930306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:32.929742 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577f596567-mss5z"] Oct 29 23:24:32.930306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:32.929931 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-577f596567-mss5z" podUID="be79ca34-64ac-4752-bba8-be72842d52d8" containerName="manager" containerID="cri-o://f9a84b645b195ca4782cf6e48d9b74b70a1c4874ba3917603d7a1db052eccb52" gracePeriod=30 Oct 29 23:24:33.193920 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.193893 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-mss5z" Oct 29 23:24:33.377412 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.377336 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be79ca34-64ac-4752-bba8-be72842d52d8-kube-api-access-qdfp2\" (UniqueName: \"kubernetes.io/projected/be79ca34-64ac-4752-bba8-be72842d52d8-kube-api-access-qdfp2\") pod \"be79ca34-64ac-4752-bba8-be72842d52d8\" (UID: \"be79ca34-64ac-4752-bba8-be72842d52d8\") " Oct 29 23:24:33.377412 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.377399 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/be79ca34-64ac-4752-bba8-be72842d52d8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/be79ca34-64ac-4752-bba8-be72842d52d8-metrics-certs\") pod \"be79ca34-64ac-4752-bba8-be72842d52d8\" (UID: \"be79ca34-64ac-4752-bba8-be72842d52d8\") " Oct 29 23:24:33.377613 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.377431 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/be79ca34-64ac-4752-bba8-be72842d52d8-webhook-cert\" (UniqueName: \"kubernetes.io/secret/be79ca34-64ac-4752-bba8-be72842d52d8-webhook-cert\") pod \"be79ca34-64ac-4752-bba8-be72842d52d8\" (UID: \"be79ca34-64ac-4752-bba8-be72842d52d8\") " Oct 29 23:24:33.379967 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.379926 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be79ca34-64ac-4752-bba8-be72842d52d8-webhook-cert" pod "be79ca34-64ac-4752-bba8-be72842d52d8" (UID: "be79ca34-64ac-4752-bba8-be72842d52d8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:24:33.380120 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.380029 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be79ca34-64ac-4752-bba8-be72842d52d8-metrics-certs" pod "be79ca34-64ac-4752-bba8-be72842d52d8" (UID: "be79ca34-64ac-4752-bba8-be72842d52d8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:24:33.380495 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.380470 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be79ca34-64ac-4752-bba8-be72842d52d8-kube-api-access-qdfp2" pod "be79ca34-64ac-4752-bba8-be72842d52d8" (UID: "be79ca34-64ac-4752-bba8-be72842d52d8"). InnerVolumeSpecName "kube-api-access-qdfp2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:24:33.478107 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.478068 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdfp2\" (UniqueName: \"kubernetes.io/projected/be79ca34-64ac-4752-bba8-be72842d52d8-kube-api-access-qdfp2\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:24:33.478107 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.478104 5056 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be79ca34-64ac-4752-bba8-be72842d52d8-metrics-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:24:33.478107 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.478117 5056 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be79ca34-64ac-4752-bba8-be72842d52d8-webhook-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:24:33.912186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.912130 5056 generic.go:356] "Generic (PLEG): container finished" podID="be79ca34-64ac-4752-bba8-be72842d52d8" containerID="f9a84b645b195ca4782cf6e48d9b74b70a1c4874ba3917603d7a1db052eccb52" exitCode=0 Oct 29 23:24:33.912721 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.912193 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-mss5z" Oct 29 23:24:33.912721 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.912198 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-mss5z" event={"ID":"be79ca34-64ac-4752-bba8-be72842d52d8","Type":"ContainerDied","Data":"f9a84b645b195ca4782cf6e48d9b74b70a1c4874ba3917603d7a1db052eccb52"} Oct 29 23:24:33.912721 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.912252 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-mss5z" event={"ID":"be79ca34-64ac-4752-bba8-be72842d52d8","Type":"ContainerDied","Data":"bcfeec58bceb29d0efe9978bb053954b14b442ee6596e93ad21926283058b9e0"} Oct 29 23:24:33.912721 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.912283 5056 scope.go:122] "RemoveContainer" containerID="f9a84b645b195ca4782cf6e48d9b74b70a1c4874ba3917603d7a1db052eccb52" Oct 29 23:24:33.922660 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.922643 5056 scope.go:122] "RemoveContainer" containerID="f9a84b645b195ca4782cf6e48d9b74b70a1c4874ba3917603d7a1db052eccb52" Oct 29 23:24:33.922935 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:24:33.922904 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9a84b645b195ca4782cf6e48d9b74b70a1c4874ba3917603d7a1db052eccb52\": container with ID starting with f9a84b645b195ca4782cf6e48d9b74b70a1c4874ba3917603d7a1db052eccb52 not found: ID does not exist" containerID="f9a84b645b195ca4782cf6e48d9b74b70a1c4874ba3917603d7a1db052eccb52" Oct 29 23:24:33.923041 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.922945 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9a84b645b195ca4782cf6e48d9b74b70a1c4874ba3917603d7a1db052eccb52"} err="failed to get container status \"f9a84b645b195ca4782cf6e48d9b74b70a1c4874ba3917603d7a1db052eccb52\": rpc error: code = NotFound desc = could not find container \"f9a84b645b195ca4782cf6e48d9b74b70a1c4874ba3917603d7a1db052eccb52\": container with ID starting with f9a84b645b195ca4782cf6e48d9b74b70a1c4874ba3917603d7a1db052eccb52 not found: ID does not exist" Oct 29 23:24:33.966399 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.966364 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577f596567-mss5z"] Oct 29 23:24:33.979728 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:33.979691 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-577f596567-mss5z"] Oct 29 23:24:34.771123 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:34.771087 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be79ca34-64ac-4752-bba8-be72842d52d8" path="/var/lib/kubelet/pods/be79ca34-64ac-4752-bba8-be72842d52d8/volumes" Oct 29 23:24:35.924656 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:35.924623 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" event={"ID":"7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23","Type":"ContainerStarted","Data":"41f977674ff6c65b610aac995d57b808341c11a8c462957c8474a218c74a5a19"} Oct 29 23:24:35.925042 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:35.924761 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" Oct 29 23:24:35.968128 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:35.968075 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" podStartSLOduration=2.043601817 podStartE2EDuration="11.968065399s" totalImagesPullingTime="9.924463582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:24:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:24:35.967876458 +0000 UTC m=+703.334624090" watchObservedRunningTime="2026-10-29 23:24:35.968065399 +0000 UTC m=+703.334813030" Oct 29 23:24:43.757786 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:43.757727 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:24:46.935406 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:46.935374 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" Oct 29 23:24:52.414806 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:24:52.414763 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:24:55.583320 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:55.583281 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:24:55.747917 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:55.747886 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:24:56.974514 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:56.974482 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:24:57.107162 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.107127 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-dg55x"] Oct 29 23:24:57.107505 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.107493 5056 cpu_manager.go:414] "containerMap: removing container" podUID="be79ca34-64ac-4752-bba8-be72842d52d8" containerName="manager" Oct 29 23:24:57.107559 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.107507 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be79ca34-64ac-4752-bba8-be72842d52d8" containerName="manager" Oct 29 23:24:57.107606 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.107597 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="be79ca34-64ac-4752-bba8-be72842d52d8" containerName="manager" Oct 29 23:24:57.113274 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.113248 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:24:57.133491 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.133462 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:24:57.145095 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.145067 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Oct 29 23:24:57.145240 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.145171 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Oct 29 23:24:57.147411 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.147387 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-dg55x"] Oct 29 23:24:57.182813 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.182789 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95874d50-ec6c-4370-919a-47568d8e8834-openshift-service-ca\") pod \"maas-api-5c48877c75-dg55x\" (UID: \"95874d50-ec6c-4370-919a-47568d8e8834\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:24:57.182965 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.182817 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95874d50-ec6c-4370-919a-47568d8e8834-metrics-certs\") pod \"maas-api-5c48877c75-dg55x\" (UID: \"95874d50-ec6c-4370-919a-47568d8e8834\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:24:57.182965 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.182844 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95874d50-ec6c-4370-919a-47568d8e8834-maas-api-tls\") pod \"maas-api-5c48877c75-dg55x\" (UID: \"95874d50-ec6c-4370-919a-47568d8e8834\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:24:57.182965 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.182859 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdzmq\" (UniqueName: \"kubernetes.io/projected/95874d50-ec6c-4370-919a-47568d8e8834-kube-api-access-kdzmq\") pod \"maas-api-5c48877c75-dg55x\" (UID: \"95874d50-ec6c-4370-919a-47568d8e8834\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:24:57.284382 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.284303 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95874d50-ec6c-4370-919a-47568d8e8834-openshift-service-ca\") pod \"maas-api-5c48877c75-dg55x\" (UID: \"95874d50-ec6c-4370-919a-47568d8e8834\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:24:57.284382 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.284337 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95874d50-ec6c-4370-919a-47568d8e8834-metrics-certs\") pod \"maas-api-5c48877c75-dg55x\" (UID: \"95874d50-ec6c-4370-919a-47568d8e8834\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:24:57.284382 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.284368 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95874d50-ec6c-4370-919a-47568d8e8834-maas-api-tls\") pod \"maas-api-5c48877c75-dg55x\" (UID: \"95874d50-ec6c-4370-919a-47568d8e8834\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:24:57.284382 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.284383 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdzmq\" (UniqueName: \"kubernetes.io/projected/95874d50-ec6c-4370-919a-47568d8e8834-kube-api-access-kdzmq\") pod \"maas-api-5c48877c75-dg55x\" (UID: \"95874d50-ec6c-4370-919a-47568d8e8834\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:24:57.285157 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.285132 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95874d50-ec6c-4370-919a-47568d8e8834-openshift-service-ca\") pod \"maas-api-5c48877c75-dg55x\" (UID: \"95874d50-ec6c-4370-919a-47568d8e8834\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:24:57.286750 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.286728 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95874d50-ec6c-4370-919a-47568d8e8834-maas-api-tls\") pod \"maas-api-5c48877c75-dg55x\" (UID: \"95874d50-ec6c-4370-919a-47568d8e8834\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:24:57.286969 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.286952 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95874d50-ec6c-4370-919a-47568d8e8834-metrics-certs\") pod \"maas-api-5c48877c75-dg55x\" (UID: \"95874d50-ec6c-4370-919a-47568d8e8834\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:24:57.308701 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.308672 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdzmq\" (UniqueName: \"kubernetes.io/projected/95874d50-ec6c-4370-919a-47568d8e8834-kube-api-access-kdzmq\") pod \"maas-api-5c48877c75-dg55x\" (UID: \"95874d50-ec6c-4370-919a-47568d8e8834\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:24:57.436381 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.436344 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:24:57.734914 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:57.734888 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-dg55x"] Oct 29 23:24:57.737206 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:24:57.737174 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95874d50_ec6c_4370_919a_47568d8e8834.slice/crio-aa4255a1b9fb5968ff4d79c87a3acbf60b0d67ad5cba63c023742cf6d84e5bc8 WatchSource:0}: Error finding container aa4255a1b9fb5968ff4d79c87a3acbf60b0d67ad5cba63c023742cf6d84e5bc8: Status 404 returned error can't find the container with id aa4255a1b9fb5968ff4d79c87a3acbf60b0d67ad5cba63c023742cf6d84e5bc8 Oct 29 23:24:58.027018 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:24:58.026933 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" event={"ID":"95874d50-ec6c-4370-919a-47568d8e8834","Type":"ContainerStarted","Data":"aa4255a1b9fb5968ff4d79c87a3acbf60b0d67ad5cba63c023742cf6d84e5bc8"} Oct 29 23:25:00.038329 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:25:00.038292 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" event={"ID":"95874d50-ec6c-4370-919a-47568d8e8834","Type":"ContainerStarted","Data":"86db50973e1d184bd61edf3ecf9ff686c1bec64e753604ae922cc033a9939e7d"} Oct 29 23:25:00.038773 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:25:00.038391 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:25:00.086696 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:25:00.086649 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" podStartSLOduration=1.737479016 podStartE2EDuration="3.086635197s" totalImagesPullingTime="1.349156181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:24:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:25:00.085558178 +0000 UTC m=+727.452305810" watchObservedRunningTime="2026-10-29 23:25:00.086635197 +0000 UTC m=+727.453382828" Oct 29 23:25:06.055308 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:25:06.055254 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:25:22.414672 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:25:22.414630 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:25:41.414061 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:25:41.413990 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:25:41.682430 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:25:41.682353 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:25:43.881886 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:25:43.881847 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:25:52.415073 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:25:52.415035 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:25:58.022995 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:25:58.022952 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:25:59.913733 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:25:59.913666 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:02.778477 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:02.778435 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:04.001937 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.001902 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp"] Oct 29 23:26:04.006004 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.005977 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.028437 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.028404 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Oct 29 23:26:04.037362 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.037321 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k87bg\"" Oct 29 23:26:04.037362 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.037348 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Oct 29 23:26:04.037362 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.037356 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Oct 29 23:26:04.039131 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.039091 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp"] Oct 29 23:26:04.089569 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.089534 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee561e87-9076-42d1-902f-12dc84af0988-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.089755 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.089578 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee561e87-9076-42d1-902f-12dc84af0988-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.089755 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.089606 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee561e87-9076-42d1-902f-12dc84af0988-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.089755 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.089706 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee561e87-9076-42d1-902f-12dc84af0988-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.089857 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.089763 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k22dg\" (UniqueName: \"kubernetes.io/projected/ee561e87-9076-42d1-902f-12dc84af0988-kube-api-access-k22dg\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.089900 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.089857 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee561e87-9076-42d1-902f-12dc84af0988-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.190637 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.190599 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee561e87-9076-42d1-902f-12dc84af0988-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.190803 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.190660 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee561e87-9076-42d1-902f-12dc84af0988-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.190803 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.190778 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee561e87-9076-42d1-902f-12dc84af0988-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.190897 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.190812 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee561e87-9076-42d1-902f-12dc84af0988-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.190897 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.190878 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k22dg\" (UniqueName: \"kubernetes.io/projected/ee561e87-9076-42d1-902f-12dc84af0988-kube-api-access-k22dg\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.190976 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.190929 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee561e87-9076-42d1-902f-12dc84af0988-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.191230 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.191203 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee561e87-9076-42d1-902f-12dc84af0988-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.191366 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.191203 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee561e87-9076-42d1-902f-12dc84af0988-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.191366 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.191214 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee561e87-9076-42d1-902f-12dc84af0988-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.192902 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.192874 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee561e87-9076-42d1-902f-12dc84af0988-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.193310 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.193287 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee561e87-9076-42d1-902f-12dc84af0988-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.219291 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.219242 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:04.223615 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.223589 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k22dg\" (UniqueName: \"kubernetes.io/projected/ee561e87-9076-42d1-902f-12dc84af0988-kube-api-access-k22dg\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp\" (UID: \"ee561e87-9076-42d1-902f-12dc84af0988\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.322741 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.322643 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:04.629116 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.629075 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp"] Oct 29 23:26:04.631430 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:26:04.631405 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee561e87_9076_42d1_902f_12dc84af0988.slice/crio-67aa038ae70554a69ce122afc669090055446d3dd435e79ef9e6409d19973238 WatchSource:0}: Error finding container 67aa038ae70554a69ce122afc669090055446d3dd435e79ef9e6409d19973238: Status 404 returned error can't find the container with id 67aa038ae70554a69ce122afc669090055446d3dd435e79ef9e6409d19973238 Oct 29 23:26:04.632738 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:04.632720 5056 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 29 23:26:05.363281 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:05.363229 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" event={"ID":"ee561e87-9076-42d1-902f-12dc84af0988","Type":"ContainerStarted","Data":"67aa038ae70554a69ce122afc669090055446d3dd435e79ef9e6409d19973238"} Oct 29 23:26:06.213858 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:06.213821 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:06.369669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:06.369633 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" event={"ID":"ee561e87-9076-42d1-902f-12dc84af0988","Type":"ContainerStarted","Data":"da72f9e470320850956bf0ee5794275bb21979a5db8e560d40f26aa617b14208"} Oct 29 23:26:06.415113 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:06.415060 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" podStartSLOduration=2.319787848 podStartE2EDuration="3.415045776s" totalImagesPullingTime="1.095257928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:26:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:26:06.413684665 +0000 UTC m=+793.780432296" watchObservedRunningTime="2026-10-29 23:26:06.415045776 +0000 UTC m=+793.781793407" Oct 29 23:26:07.323244 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:07.323204 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:08.410459 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.410427 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q"] Oct 29 23:26:08.414121 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.414097 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.429992 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.429970 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Oct 29 23:26:08.445452 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.445427 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q"] Oct 29 23:26:08.533233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.533204 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae66d7aa-6da8-48c2-992c-73bc167130a1-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.533233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.533238 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twng8\" (UniqueName: \"kubernetes.io/projected/ae66d7aa-6da8-48c2-992c-73bc167130a1-kube-api-access-twng8\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.533463 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.533307 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae66d7aa-6da8-48c2-992c-73bc167130a1-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.533463 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.533451 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae66d7aa-6da8-48c2-992c-73bc167130a1-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.533529 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.533514 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae66d7aa-6da8-48c2-992c-73bc167130a1-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.533582 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.533567 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae66d7aa-6da8-48c2-992c-73bc167130a1-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.579010 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.578979 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:08.634162 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.634129 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae66d7aa-6da8-48c2-992c-73bc167130a1-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.634162 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.634164 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twng8\" (UniqueName: \"kubernetes.io/projected/ae66d7aa-6da8-48c2-992c-73bc167130a1-kube-api-access-twng8\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.634366 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.634214 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae66d7aa-6da8-48c2-992c-73bc167130a1-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.634366 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.634276 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae66d7aa-6da8-48c2-992c-73bc167130a1-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.634451 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.634389 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae66d7aa-6da8-48c2-992c-73bc167130a1-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.634451 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.634438 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae66d7aa-6da8-48c2-992c-73bc167130a1-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.634675 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.634649 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae66d7aa-6da8-48c2-992c-73bc167130a1-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.634821 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.634809 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae66d7aa-6da8-48c2-992c-73bc167130a1-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.634881 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.634821 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae66d7aa-6da8-48c2-992c-73bc167130a1-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.636763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.636741 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae66d7aa-6da8-48c2-992c-73bc167130a1-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.636866 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.636805 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae66d7aa-6da8-48c2-992c-73bc167130a1-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.660622 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.660559 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-twng8\" (UniqueName: \"kubernetes.io/projected/ae66d7aa-6da8-48c2-992c-73bc167130a1-kube-api-access-twng8\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-xwh8q\" (UID: \"ae66d7aa-6da8-48c2-992c-73bc167130a1\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:08.727657 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:08.727627 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:09.031416 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:09.031370 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q"] Oct 29 23:26:09.033181 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:26:09.033153 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae66d7aa_6da8_48c2_992c_73bc167130a1.slice/crio-1fd943c9c1dfbd608703d9fe51376dfb1a172f0afcb9f29f2e27b79b90f65577 WatchSource:0}: Error finding container 1fd943c9c1dfbd608703d9fe51376dfb1a172f0afcb9f29f2e27b79b90f65577: Status 404 returned error can't find the container with id 1fd943c9c1dfbd608703d9fe51376dfb1a172f0afcb9f29f2e27b79b90f65577 Oct 29 23:26:09.385334 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:09.385296 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" event={"ID":"ae66d7aa-6da8-48c2-992c-73bc167130a1","Type":"ContainerStarted","Data":"4d6e6a7b26b21024166445e6a672afba971d7aaf5532038e616f04cabdcbabc2"} Oct 29 23:26:09.385513 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:09.385358 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" event={"ID":"ae66d7aa-6da8-48c2-992c-73bc167130a1","Type":"ContainerStarted","Data":"1fd943c9c1dfbd608703d9fe51376dfb1a172f0afcb9f29f2e27b79b90f65577"} Oct 29 23:26:09.433279 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:09.433211 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" podStartSLOduration=1.219138156 podStartE2EDuration="1.433196584s" totalImagesPullingTime="214.058428ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:26:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:26:09.431705405 +0000 UTC m=+796.798453038" watchObservedRunningTime="2026-10-29 23:26:09.433196584 +0000 UTC m=+796.799944214" Oct 29 23:26:09.728066 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:09.728034 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:10.860579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:10.860547 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft"] Oct 29 23:26:10.864417 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:10.864392 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:10.887461 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:10.887435 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft"] Oct 29 23:26:10.896250 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:10.896228 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Oct 29 23:26:10.957622 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:10.957592 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36624910-931c-4b68-a211-ec47fa0beebc-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:10.957793 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:10.957630 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36624910-931c-4b68-a211-ec47fa0beebc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:10.957793 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:10.957652 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36624910-931c-4b68-a211-ec47fa0beebc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:10.957793 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:10.957777 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36624910-931c-4b68-a211-ec47fa0beebc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:10.957897 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:10.957807 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tzkc\" (UniqueName: \"kubernetes.io/projected/36624910-931c-4b68-a211-ec47fa0beebc-kube-api-access-2tzkc\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:10.957897 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:10.957839 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36624910-931c-4b68-a211-ec47fa0beebc-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:10.978832 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:10.978801 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:11.058634 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:11.058597 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36624910-931c-4b68-a211-ec47fa0beebc-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:11.058969 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:11.058657 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36624910-931c-4b68-a211-ec47fa0beebc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:11.058969 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:11.058683 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36624910-931c-4b68-a211-ec47fa0beebc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:11.058969 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:11.058742 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36624910-931c-4b68-a211-ec47fa0beebc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:11.058969 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:11.058789 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tzkc\" (UniqueName: \"kubernetes.io/projected/36624910-931c-4b68-a211-ec47fa0beebc-kube-api-access-2tzkc\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:11.058969 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:11.058843 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36624910-931c-4b68-a211-ec47fa0beebc-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:11.059232 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:11.059211 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36624910-931c-4b68-a211-ec47fa0beebc-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:11.059296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:11.059228 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36624910-931c-4b68-a211-ec47fa0beebc-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:11.059296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:11.059280 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36624910-931c-4b68-a211-ec47fa0beebc-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:11.061101 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:11.061081 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36624910-931c-4b68-a211-ec47fa0beebc-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:11.061285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:11.061244 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36624910-931c-4b68-a211-ec47fa0beebc-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:11.106240 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:11.106206 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tzkc\" (UniqueName: \"kubernetes.io/projected/36624910-931c-4b68-a211-ec47fa0beebc-kube-api-access-2tzkc\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-7pbft\" (UID: \"36624910-931c-4b68-a211-ec47fa0beebc\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:11.177844 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:11.177807 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:11.479199 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:11.479173 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft"] Oct 29 23:26:11.481099 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:26:11.481074 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36624910_931c_4b68_a211_ec47fa0beebc.slice/crio-f441fcd4338261fd3f47899d377c882a29fcc25415b4d00b69f74b56a3907651 WatchSource:0}: Error finding container f441fcd4338261fd3f47899d377c882a29fcc25415b4d00b69f74b56a3907651: Status 404 returned error can't find the container with id f441fcd4338261fd3f47899d377c882a29fcc25415b4d00b69f74b56a3907651 Oct 29 23:26:12.401059 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:12.401021 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" event={"ID":"36624910-931c-4b68-a211-ec47fa0beebc","Type":"ContainerStarted","Data":"060ae7d33ee39f84d1364771aa272e38fa2d85f2f5e4110bc3928290c0cc55be"} Oct 29 23:26:12.401579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:12.401123 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" event={"ID":"36624910-931c-4b68-a211-ec47fa0beebc","Type":"ContainerStarted","Data":"f441fcd4338261fd3f47899d377c882a29fcc25415b4d00b69f74b56a3907651"} Oct 29 23:26:12.446533 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:12.446485 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" podStartSLOduration=2.216684535 podStartE2EDuration="2.44647165s" totalImagesPullingTime="229.787115ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:26:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:26:12.446388764 +0000 UTC m=+799.813136394" watchObservedRunningTime="2026-10-29 23:26:12.44647165 +0000 UTC m=+799.813219282" Oct 29 23:26:13.178855 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:13.178827 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:14.322938 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:14.322898 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:14.335721 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:14.335698 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:14.421764 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:14.421730 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp" Oct 29 23:26:17.581652 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:17.581624 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:17.740581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:17.740544 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:18.728541 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:18.728503 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:18.741371 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:18.741349 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:19.445902 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:19.445865 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-xwh8q" Oct 29 23:26:19.807523 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:19.807436 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg"] Oct 29 23:26:19.811818 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:19.811758 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" Oct 29 23:26:19.832308 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:19.832277 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\"" Oct 29 23:26:19.832616 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:19.832600 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-nm8nz\"" Oct 29 23:26:19.832675 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:19.832638 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\"" Oct 29 23:26:19.840149 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:19.840124 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7834a7b7-5ce3-4230-a154-76174a00e4f2-metrics-certs\") pod \"ai-gateway-controller-7cb8f6bb64-p2tzg\" (UID: \"7834a7b7-5ce3-4230-a154-76174a00e4f2\") " pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" Oct 29 23:26:19.840233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:19.840181 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxr4t\" (UniqueName: \"kubernetes.io/projected/7834a7b7-5ce3-4230-a154-76174a00e4f2-kube-api-access-mxr4t\") pod \"ai-gateway-controller-7cb8f6bb64-p2tzg\" (UID: \"7834a7b7-5ce3-4230-a154-76174a00e4f2\") " pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" Oct 29 23:26:19.847356 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:19.847329 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg"] Oct 29 23:26:19.883296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:19.883241 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:19.940828 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:19.940779 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7834a7b7-5ce3-4230-a154-76174a00e4f2-metrics-certs\") pod \"ai-gateway-controller-7cb8f6bb64-p2tzg\" (UID: \"7834a7b7-5ce3-4230-a154-76174a00e4f2\") " pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" Oct 29 23:26:19.941032 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:19.940864 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxr4t\" (UniqueName: \"kubernetes.io/projected/7834a7b7-5ce3-4230-a154-76174a00e4f2-kube-api-access-mxr4t\") pod \"ai-gateway-controller-7cb8f6bb64-p2tzg\" (UID: \"7834a7b7-5ce3-4230-a154-76174a00e4f2\") " pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" Oct 29 23:26:19.943551 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:19.943519 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7834a7b7-5ce3-4230-a154-76174a00e4f2-metrics-certs\") pod \"ai-gateway-controller-7cb8f6bb64-p2tzg\" (UID: \"7834a7b7-5ce3-4230-a154-76174a00e4f2\") " pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" Oct 29 23:26:19.965813 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:19.965777 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxr4t\" (UniqueName: \"kubernetes.io/projected/7834a7b7-5ce3-4230-a154-76174a00e4f2-kube-api-access-mxr4t\") pod \"ai-gateway-controller-7cb8f6bb64-p2tzg\" (UID: \"7834a7b7-5ce3-4230-a154-76174a00e4f2\") " pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" Oct 29 23:26:20.043682 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:20.043647 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:20.127542 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:20.127467 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" Oct 29 23:26:20.426963 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:20.426936 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg"] Oct 29 23:26:20.428966 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:26:20.428935 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7834a7b7_5ce3_4230_a154_76174a00e4f2.slice/crio-99693a9a4cbce5e2bc9060c2507323f13cf00855344b616beeb5f29aec8c3c66 WatchSource:0}: Error finding container 99693a9a4cbce5e2bc9060c2507323f13cf00855344b616beeb5f29aec8c3c66: Status 404 returned error can't find the container with id 99693a9a4cbce5e2bc9060c2507323f13cf00855344b616beeb5f29aec8c3c66 Oct 29 23:26:20.440890 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:20.440862 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" event={"ID":"7834a7b7-5ce3-4230-a154-76174a00e4f2","Type":"ContainerStarted","Data":"99693a9a4cbce5e2bc9060c2507323f13cf00855344b616beeb5f29aec8c3c66"} Oct 29 23:26:21.178945 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:21.178905 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:21.195841 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:21.195814 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:21.465084 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:21.464804 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-7pbft" Oct 29 23:26:22.417431 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:26:22.417396 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:26:23.464890 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:23.464851 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" event={"ID":"7834a7b7-5ce3-4230-a154-76174a00e4f2","Type":"ContainerStarted","Data":"31bc9ce3358d8439a952e501d564de49dbff7462d0cfe6d5d437e4c622751d6b"} Oct 29 23:26:23.465340 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:23.465023 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" Oct 29 23:26:23.511371 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:23.511324 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" podStartSLOduration=2.068415286 podStartE2EDuration="4.511311657s" totalImagesPullingTime="2.442896371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:26:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:26:23.509228613 +0000 UTC m=+810.875976245" watchObservedRunningTime="2026-10-29 23:26:23.511311657 +0000 UTC m=+810.878059288" Oct 29 23:26:25.498474 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.498436 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:25.581004 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.580970 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-4xjsb"] Oct 29 23:26:25.584749 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.584728 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-4xjsb" Oct 29 23:26:25.592519 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.592496 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-rfjdl"] Oct 29 23:26:25.596388 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.596369 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-rfjdl" Oct 29 23:26:25.601120 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.601095 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-v6ptr\"" Oct 29 23:26:25.610227 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.610203 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-4xjsb"] Oct 29 23:26:25.618956 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.618933 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Oct 29 23:26:25.643044 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.643015 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-rfjdl"] Oct 29 23:26:25.680206 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.680172 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:25.699618 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.699582 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1064c092-6e9b-4ea9-a4de-4a6961a528a5-plugins-config-volume\") pod \"payload-processing-5f78687d5b-rfjdl\" (UID: \"1064c092-6e9b-4ea9-a4de-4a6961a528a5\") " pod="openshift-ingress/payload-processing-5f78687d5b-rfjdl" Oct 29 23:26:25.699785 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.699635 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27a25029-6749-4ba3-a00e-16e7bb16981b-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-4xjsb\" (UID: \"27a25029-6749-4ba3-a00e-16e7bb16981b\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-4xjsb" Oct 29 23:26:25.699785 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.699699 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5s9hw\" (UniqueName: \"kubernetes.io/projected/1064c092-6e9b-4ea9-a4de-4a6961a528a5-kube-api-access-5s9hw\") pod \"payload-processing-5f78687d5b-rfjdl\" (UID: \"1064c092-6e9b-4ea9-a4de-4a6961a528a5\") " pod="openshift-ingress/payload-processing-5f78687d5b-rfjdl" Oct 29 23:26:25.699785 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.699724 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khvxj\" (UniqueName: \"kubernetes.io/projected/27a25029-6749-4ba3-a00e-16e7bb16981b-kube-api-access-khvxj\") pod \"payload-pre-processing-7b98bb678c-4xjsb\" (UID: \"27a25029-6749-4ba3-a00e-16e7bb16981b\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-4xjsb" Oct 29 23:26:25.801254 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.801171 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1064c092-6e9b-4ea9-a4de-4a6961a528a5-plugins-config-volume\") pod \"payload-processing-5f78687d5b-rfjdl\" (UID: \"1064c092-6e9b-4ea9-a4de-4a6961a528a5\") " pod="openshift-ingress/payload-processing-5f78687d5b-rfjdl" Oct 29 23:26:25.801254 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.801219 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27a25029-6749-4ba3-a00e-16e7bb16981b-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-4xjsb\" (UID: \"27a25029-6749-4ba3-a00e-16e7bb16981b\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-4xjsb" Oct 29 23:26:25.801254 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.801238 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5s9hw\" (UniqueName: \"kubernetes.io/projected/1064c092-6e9b-4ea9-a4de-4a6961a528a5-kube-api-access-5s9hw\") pod \"payload-processing-5f78687d5b-rfjdl\" (UID: \"1064c092-6e9b-4ea9-a4de-4a6961a528a5\") " pod="openshift-ingress/payload-processing-5f78687d5b-rfjdl" Oct 29 23:26:25.801254 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.801255 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khvxj\" (UniqueName: \"kubernetes.io/projected/27a25029-6749-4ba3-a00e-16e7bb16981b-kube-api-access-khvxj\") pod \"payload-pre-processing-7b98bb678c-4xjsb\" (UID: \"27a25029-6749-4ba3-a00e-16e7bb16981b\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-4xjsb" Oct 29 23:26:25.802011 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.801986 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1064c092-6e9b-4ea9-a4de-4a6961a528a5-plugins-config-volume\") pod \"payload-processing-5f78687d5b-rfjdl\" (UID: \"1064c092-6e9b-4ea9-a4de-4a6961a528a5\") " pod="openshift-ingress/payload-processing-5f78687d5b-rfjdl" Oct 29 23:26:25.802108 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.802048 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27a25029-6749-4ba3-a00e-16e7bb16981b-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-4xjsb\" (UID: \"27a25029-6749-4ba3-a00e-16e7bb16981b\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-4xjsb" Oct 29 23:26:25.828099 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.828078 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5s9hw\" (UniqueName: \"kubernetes.io/projected/1064c092-6e9b-4ea9-a4de-4a6961a528a5-kube-api-access-5s9hw\") pod \"payload-processing-5f78687d5b-rfjdl\" (UID: \"1064c092-6e9b-4ea9-a4de-4a6961a528a5\") " pod="openshift-ingress/payload-processing-5f78687d5b-rfjdl" Oct 29 23:26:25.833103 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.833081 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khvxj\" (UniqueName: \"kubernetes.io/projected/27a25029-6749-4ba3-a00e-16e7bb16981b-kube-api-access-khvxj\") pod \"payload-pre-processing-7b98bb678c-4xjsb\" (UID: \"27a25029-6749-4ba3-a00e-16e7bb16981b\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-4xjsb" Oct 29 23:26:25.899231 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.899200 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-4xjsb" Oct 29 23:26:25.909347 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:25.909320 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5f78687d5b-rfjdl" Oct 29 23:26:26.207723 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:26.207690 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-4xjsb"] Oct 29 23:26:26.213604 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:26:26.213575 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27a25029_6749_4ba3_a00e_16e7bb16981b.slice/crio-3fbab4b8e48bf98d5bde6c4c724356e2f9dbeb1e217c8a809dbb65f3a4328e06 WatchSource:0}: Error finding container 3fbab4b8e48bf98d5bde6c4c724356e2f9dbeb1e217c8a809dbb65f3a4328e06: Status 404 returned error can't find the container with id 3fbab4b8e48bf98d5bde6c4c724356e2f9dbeb1e217c8a809dbb65f3a4328e06 Oct 29 23:26:26.224518 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:26.224495 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5f78687d5b-rfjdl"] Oct 29 23:26:26.225747 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:26:26.225724 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1064c092_6e9b_4ea9_a4de_4a6961a528a5.slice/crio-1172d790832608b223383c70ffd47de25c1b756d30583ee2e057b21a9c933ada WatchSource:0}: Error finding container 1172d790832608b223383c70ffd47de25c1b756d30583ee2e057b21a9c933ada: Status 404 returned error can't find the container with id 1172d790832608b223383c70ffd47de25c1b756d30583ee2e057b21a9c933ada Oct 29 23:26:26.480221 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:26.480128 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-rfjdl" event={"ID":"1064c092-6e9b-4ea9-a4de-4a6961a528a5","Type":"ContainerStarted","Data":"1172d790832608b223383c70ffd47de25c1b756d30583ee2e057b21a9c933ada"} Oct 29 23:26:26.481403 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:26.481372 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-4xjsb" event={"ID":"27a25029-6749-4ba3-a00e-16e7bb16981b","Type":"ContainerStarted","Data":"3fbab4b8e48bf98d5bde6c4c724356e2f9dbeb1e217c8a809dbb65f3a4328e06"} Oct 29 23:26:29.499415 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:29.499376 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5f78687d5b-rfjdl" event={"ID":"1064c092-6e9b-4ea9-a4de-4a6961a528a5","Type":"ContainerStarted","Data":"234d05d2bc5cc829d6ef3430696229f48f2f92e8c3bdc532046381869603e9a5"} Oct 29 23:26:29.499892 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:29.499527 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5f78687d5b-rfjdl" Oct 29 23:26:29.500985 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:29.500960 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-4xjsb" event={"ID":"27a25029-6749-4ba3-a00e-16e7bb16981b","Type":"ContainerStarted","Data":"e341b95dc6bad79a61cb9b7aebfd0798291884e43e8d0e5cb1132ee587e306d5"} Oct 29 23:26:29.501107 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:29.501091 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-4xjsb" Oct 29 23:26:29.544963 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:29.544919 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5f78687d5b-rfjdl" podStartSLOduration=2.149236285 podStartE2EDuration="4.544905908s" totalImagesPullingTime="2.395669623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:26:29.543761423 +0000 UTC m=+816.910509055" watchObservedRunningTime="2026-10-29 23:26:29.544905908 +0000 UTC m=+816.911653538" Oct 29 23:26:29.601307 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:29.601239 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-4xjsb" podStartSLOduration=2.2293512890000002 podStartE2EDuration="4.601225627s" totalImagesPullingTime="2.371874338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:26:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:26:29.600148455 +0000 UTC m=+816.966896088" watchObservedRunningTime="2026-10-29 23:26:29.601225627 +0000 UTC m=+816.967973257" Oct 29 23:26:34.472223 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:34.472189 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" Oct 29 23:26:36.936065 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:36.936034 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2"] Oct 29 23:26:36.942345 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:36.942324 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:36.958377 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:36.958352 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-f5fmz\"" Oct 29 23:26:36.973914 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:36.973884 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2"] Oct 29 23:26:37.108275 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.108230 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a27239f9-531e-4f44-be00-22c11b243a3a-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.108446 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.108281 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a27239f9-531e-4f44-be00-22c11b243a3a-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.108446 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.108302 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a27239f9-531e-4f44-be00-22c11b243a3a-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.108545 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.108452 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4jmw\" (UniqueName: \"kubernetes.io/projected/a27239f9-531e-4f44-be00-22c11b243a3a-kube-api-access-v4jmw\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.108545 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.108484 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a27239f9-531e-4f44-be00-22c11b243a3a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.108613 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.108557 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a27239f9-531e-4f44-be00-22c11b243a3a-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.108613 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.108598 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a27239f9-531e-4f44-be00-22c11b243a3a-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.108671 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.108632 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a27239f9-531e-4f44-be00-22c11b243a3a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.108709 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.108699 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a27239f9-531e-4f44-be00-22c11b243a3a-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.209578 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.209479 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4jmw\" (UniqueName: \"kubernetes.io/projected/a27239f9-531e-4f44-be00-22c11b243a3a-kube-api-access-v4jmw\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.209578 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.209519 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a27239f9-531e-4f44-be00-22c11b243a3a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.209578 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.209568 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a27239f9-531e-4f44-be00-22c11b243a3a-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.209859 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.209591 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a27239f9-531e-4f44-be00-22c11b243a3a-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.209859 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.209615 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a27239f9-531e-4f44-be00-22c11b243a3a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.209859 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.209655 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a27239f9-531e-4f44-be00-22c11b243a3a-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.209859 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.209724 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a27239f9-531e-4f44-be00-22c11b243a3a-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.209859 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.209750 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a27239f9-531e-4f44-be00-22c11b243a3a-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.209859 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.209774 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a27239f9-531e-4f44-be00-22c11b243a3a-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.210150 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.210044 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a27239f9-531e-4f44-be00-22c11b243a3a-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.210150 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.210083 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a27239f9-531e-4f44-be00-22c11b243a3a-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.210150 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.210122 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a27239f9-531e-4f44-be00-22c11b243a3a-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.210317 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.210250 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a27239f9-531e-4f44-be00-22c11b243a3a-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.210612 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.210595 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a27239f9-531e-4f44-be00-22c11b243a3a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.212106 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.212079 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a27239f9-531e-4f44-be00-22c11b243a3a-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.212201 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.212127 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a27239f9-531e-4f44-be00-22c11b243a3a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.234083 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.234048 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a27239f9-531e-4f44-be00-22c11b243a3a-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.234182 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.234167 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4jmw\" (UniqueName: \"kubernetes.io/projected/a27239f9-531e-4f44-be00-22c11b243a3a-kube-api-access-v4jmw\") pod \"maas-default-gateway-openshift-default-746ff8f49f-9k9s2\" (UID: \"a27239f9-531e-4f44-be00-22c11b243a3a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.256722 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.256692 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:37.577833 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:37.577798 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2"] Oct 29 23:26:37.579212 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:26:37.579189 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda27239f9_531e_4f44_be00_22c11b243a3a.slice/crio-8a07335bcc560ce15cd24cc84946757de9309deb8f47f7f257583fed3a06c6c9 WatchSource:0}: Error finding container 8a07335bcc560ce15cd24cc84946757de9309deb8f47f7f257583fed3a06c6c9: Status 404 returned error can't find the container with id 8a07335bcc560ce15cd24cc84946757de9309deb8f47f7f257583fed3a06c6c9 Oct 29 23:26:38.547151 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:38.547109 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" event={"ID":"a27239f9-531e-4f44-be00-22c11b243a3a","Type":"ContainerStarted","Data":"8a07335bcc560ce15cd24cc84946757de9309deb8f47f7f257583fed3a06c6c9"} Oct 29 23:26:40.115414 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:40.115377 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:26:40.115720 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:40.115445 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:26:40.115720 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:40.115474 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:26:40.509470 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:40.509440 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5f78687d5b-rfjdl" Oct 29 23:26:40.509636 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:40.509511 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-4xjsb" Oct 29 23:26:40.588481 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:40.588444 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" event={"ID":"a27239f9-531e-4f44-be00-22c11b243a3a","Type":"ContainerStarted","Data":"14daf700c18b46925618ae2517439097a329cd61119d93ca47e3495bbfcc7506"} Oct 29 23:26:41.257766 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:41.257724 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:41.259293 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:41.259247 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-9k9s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Oct 29 23:26:41.259420 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:41.259326 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" podUID="a27239f9-531e-4f44-be00-22c11b243a3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Oct 29 23:26:42.257593 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:42.257552 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-9k9s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Oct 29 23:26:42.257797 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:42.257620 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" podUID="a27239f9-531e-4f44-be00-22c11b243a3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Oct 29 23:26:43.257483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:43.257450 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-9k9s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Oct 29 23:26:43.257634 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:43.257516 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" podUID="a27239f9-531e-4f44-be00-22c11b243a3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Oct 29 23:26:44.034875 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:44.034773 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:44.257224 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:44.257188 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-9k9s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Oct 29 23:26:44.257455 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:44.257249 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" podUID="a27239f9-531e-4f44-be00-22c11b243a3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Oct 29 23:26:45.257374 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:45.257337 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-9k9s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Oct 29 23:26:45.257797 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:45.257391 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" podUID="a27239f9-531e-4f44-be00-22c11b243a3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Oct 29 23:26:46.257545 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:46.257505 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-9k9s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Oct 29 23:26:46.258007 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:46.257562 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" podUID="a27239f9-531e-4f44-be00-22c11b243a3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Oct 29 23:26:47.258169 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:47.258132 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-9k9s2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Oct 29 23:26:47.258595 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:47.258190 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" podUID="a27239f9-531e-4f44-be00-22c11b243a3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Oct 29 23:26:48.362069 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:48.362034 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:48.362576 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:48.362366 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:48.363312 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:48.363294 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" Oct 29 23:26:48.423569 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:48.423523 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" podStartSLOduration=9.890058228000001 podStartE2EDuration="12.423511441s" totalImagesPullingTime="2.533453213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:26:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:26:40.696626441 +0000 UTC m=+828.063374072" watchObservedRunningTime="2026-10-29 23:26:48.423511441 +0000 UTC m=+835.790259071" Oct 29 23:26:49.098639 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.098602 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg"] Oct 29 23:26:49.098884 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.098849 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" podUID="7834a7b7-5ce3-4230-a154-76174a00e4f2" containerName="manager" containerID="cri-o://31bc9ce3358d8439a952e501d564de49dbff7462d0cfe6d5d437e4c622751d6b" gracePeriod=30 Oct 29 23:26:49.348273 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.348235 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" Oct 29 23:26:49.429776 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.429738 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7834a7b7-5ce3-4230-a154-76174a00e4f2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7834a7b7-5ce3-4230-a154-76174a00e4f2-metrics-certs\") pod \"7834a7b7-5ce3-4230-a154-76174a00e4f2\" (UID: \"7834a7b7-5ce3-4230-a154-76174a00e4f2\") " Oct 29 23:26:49.430243 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.429871 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7834a7b7-5ce3-4230-a154-76174a00e4f2-kube-api-access-mxr4t\" (UniqueName: \"kubernetes.io/projected/7834a7b7-5ce3-4230-a154-76174a00e4f2-kube-api-access-mxr4t\") pod \"7834a7b7-5ce3-4230-a154-76174a00e4f2\" (UID: \"7834a7b7-5ce3-4230-a154-76174a00e4f2\") " Oct 29 23:26:49.431779 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.431754 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7834a7b7-5ce3-4230-a154-76174a00e4f2-metrics-certs" pod "7834a7b7-5ce3-4230-a154-76174a00e4f2" (UID: "7834a7b7-5ce3-4230-a154-76174a00e4f2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:26:49.431852 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.431815 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7834a7b7-5ce3-4230-a154-76174a00e4f2-kube-api-access-mxr4t" pod "7834a7b7-5ce3-4230-a154-76174a00e4f2" (UID: "7834a7b7-5ce3-4230-a154-76174a00e4f2"). InnerVolumeSpecName "kube-api-access-mxr4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:26:49.531169 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.531130 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxr4t\" (UniqueName: \"kubernetes.io/projected/7834a7b7-5ce3-4230-a154-76174a00e4f2-kube-api-access-mxr4t\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:26:49.531169 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.531165 5056 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7834a7b7-5ce3-4230-a154-76174a00e4f2-metrics-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:26:49.631671 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.631629 5056 generic.go:356] "Generic (PLEG): container finished" podID="7834a7b7-5ce3-4230-a154-76174a00e4f2" containerID="31bc9ce3358d8439a952e501d564de49dbff7462d0cfe6d5d437e4c622751d6b" exitCode=0 Oct 29 23:26:49.631868 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.631683 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" Oct 29 23:26:49.631868 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.631710 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" event={"ID":"7834a7b7-5ce3-4230-a154-76174a00e4f2","Type":"ContainerDied","Data":"31bc9ce3358d8439a952e501d564de49dbff7462d0cfe6d5d437e4c622751d6b"} Oct 29 23:26:49.631868 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.631782 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg" event={"ID":"7834a7b7-5ce3-4230-a154-76174a00e4f2","Type":"ContainerDied","Data":"99693a9a4cbce5e2bc9060c2507323f13cf00855344b616beeb5f29aec8c3c66"} Oct 29 23:26:49.631868 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.631813 5056 scope.go:122] "RemoveContainer" containerID="31bc9ce3358d8439a952e501d564de49dbff7462d0cfe6d5d437e4c622751d6b" Oct 29 23:26:49.641203 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.641187 5056 scope.go:122] "RemoveContainer" containerID="31bc9ce3358d8439a952e501d564de49dbff7462d0cfe6d5d437e4c622751d6b" Oct 29 23:26:49.641499 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:26:49.641479 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31bc9ce3358d8439a952e501d564de49dbff7462d0cfe6d5d437e4c622751d6b\": container with ID starting with 31bc9ce3358d8439a952e501d564de49dbff7462d0cfe6d5d437e4c622751d6b not found: ID does not exist" containerID="31bc9ce3358d8439a952e501d564de49dbff7462d0cfe6d5d437e4c622751d6b" Oct 29 23:26:49.641560 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.641508 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31bc9ce3358d8439a952e501d564de49dbff7462d0cfe6d5d437e4c622751d6b"} err="failed to get container status \"31bc9ce3358d8439a952e501d564de49dbff7462d0cfe6d5d437e4c622751d6b\": rpc error: code = NotFound desc = could not find container \"31bc9ce3358d8439a952e501d564de49dbff7462d0cfe6d5d437e4c622751d6b\": container with ID starting with 31bc9ce3358d8439a952e501d564de49dbff7462d0cfe6d5d437e4c622751d6b not found: ID does not exist" Oct 29 23:26:49.685976 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.685896 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg"] Oct 29 23:26:49.690217 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:49.690192 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-7cb8f6bb64-p2tzg"] Oct 29 23:26:50.753022 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:50.752986 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:50.770234 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:50.770206 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7834a7b7-5ce3-4230-a154-76174a00e4f2" path="/var/lib/kubelet/pods/7834a7b7-5ce3-4230-a154-76174a00e4f2/volumes" Oct 29 23:26:51.233451 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:51.233414 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:51.983699 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:51.983665 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf"] Oct 29 23:26:51.984311 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:51.984213 5056 cpu_manager.go:414] "containerMap: removing container" podUID="7834a7b7-5ce3-4230-a154-76174a00e4f2" containerName="manager" Oct 29 23:26:51.984311 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:51.984231 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7834a7b7-5ce3-4230-a154-76174a00e4f2" containerName="manager" Oct 29 23:26:51.984380 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:51.984370 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="7834a7b7-5ce3-4230-a154-76174a00e4f2" containerName="manager" Oct 29 23:26:51.988987 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:51.988963 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.013451 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.013420 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf"] Oct 29 23:26:52.055910 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.055859 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5adc898f-40d4-4851-af95-1f400d519360-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.055910 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.055909 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5adc898f-40d4-4851-af95-1f400d519360-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.056145 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.056031 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5adc898f-40d4-4851-af95-1f400d519360-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.056145 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.056098 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5adc898f-40d4-4851-af95-1f400d519360-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.056238 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.056155 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5adc898f-40d4-4851-af95-1f400d519360-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.056325 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.056239 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5drt\" (UniqueName: \"kubernetes.io/projected/5adc898f-40d4-4851-af95-1f400d519360-kube-api-access-m5drt\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.056408 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.056383 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5adc898f-40d4-4851-af95-1f400d519360-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.056472 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.056456 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5adc898f-40d4-4851-af95-1f400d519360-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.056512 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.056488 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5adc898f-40d4-4851-af95-1f400d519360-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.157647 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.157615 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5adc898f-40d4-4851-af95-1f400d519360-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.157647 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.157651 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5adc898f-40d4-4851-af95-1f400d519360-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.157900 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.157677 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5adc898f-40d4-4851-af95-1f400d519360-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.157900 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.157701 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5adc898f-40d4-4851-af95-1f400d519360-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.157900 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.157737 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5adc898f-40d4-4851-af95-1f400d519360-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.157900 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.157888 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5adc898f-40d4-4851-af95-1f400d519360-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.158106 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.157941 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5adc898f-40d4-4851-af95-1f400d519360-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.158106 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.158018 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5drt\" (UniqueName: \"kubernetes.io/projected/5adc898f-40d4-4851-af95-1f400d519360-kube-api-access-m5drt\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.158222 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.158099 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5adc898f-40d4-4851-af95-1f400d519360-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.158222 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.158141 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5adc898f-40d4-4851-af95-1f400d519360-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.158436 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.158411 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5adc898f-40d4-4851-af95-1f400d519360-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.158559 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.158526 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5adc898f-40d4-4851-af95-1f400d519360-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.158612 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.158595 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5adc898f-40d4-4851-af95-1f400d519360-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.158836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.158819 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5adc898f-40d4-4851-af95-1f400d519360-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.160040 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.160019 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5adc898f-40d4-4851-af95-1f400d519360-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.160655 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.160631 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5adc898f-40d4-4851-af95-1f400d519360-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.187039 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.187006 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5adc898f-40d4-4851-af95-1f400d519360-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.187142 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.187114 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5drt\" (UniqueName: \"kubernetes.io/projected/5adc898f-40d4-4851-af95-1f400d519360-kube-api-access-m5drt\") pod \"maas-default-gateway-openshift-default-746ff8f49f-nbfzf\" (UID: \"5adc898f-40d4-4851-af95-1f400d519360\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.302531 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.302436 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:52.415749 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:26:52.415714 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:26:52.621398 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.621324 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf"] Oct 29 23:26:52.624653 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:26:52.624627 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5adc898f_40d4_4851_af95_1f400d519360.slice/crio-688fec377b1f6cb18164a649d22578a010fbc390f817eea782b93c8c5218bc1d WatchSource:0}: Error finding container 688fec377b1f6cb18164a649d22578a010fbc390f817eea782b93c8c5218bc1d: Status 404 returned error can't find the container with id 688fec377b1f6cb18164a649d22578a010fbc390f817eea782b93c8c5218bc1d Oct 29 23:26:52.627659 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.627628 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:26:52.627732 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.627696 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:26:52.627732 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.627724 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:26:52.650333 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.650307 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" event={"ID":"5adc898f-40d4-4851-af95-1f400d519360","Type":"ContainerStarted","Data":"688fec377b1f6cb18164a649d22578a010fbc390f817eea782b93c8c5218bc1d"} Oct 29 23:26:52.933059 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:52.933013 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:53.207908 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:53.207822 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:53.657079 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:53.657041 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" event={"ID":"5adc898f-40d4-4851-af95-1f400d519360","Type":"ContainerStarted","Data":"d4ebbed6872574d0479cff2a7af4d68803bcadb7c11d82b5633c6a9f3732afd6"} Oct 29 23:26:53.739843 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:53.739777 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" podStartSLOduration=2.7397628689999998 podStartE2EDuration="2.739762869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:26:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:26:53.738678039 +0000 UTC m=+841.105425670" watchObservedRunningTime="2026-10-29 23:26:53.739762869 +0000 UTC m=+841.106510499" Oct 29 23:26:54.302969 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:54.302937 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:26:54.304622 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:54.304590 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-nbfzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Oct 29 23:26:54.304774 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:54.304644 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" podUID="5adc898f-40d4-4851-af95-1f400d519360" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Oct 29 23:26:54.307726 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:54.307694 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:54.472756 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:54.472718 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:26:55.303198 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:55.303158 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-nbfzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Oct 29 23:26:55.303630 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:55.303219 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" podUID="5adc898f-40d4-4851-af95-1f400d519360" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Oct 29 23:26:56.303439 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:56.303398 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-nbfzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Oct 29 23:26:56.303842 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:56.303474 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" podUID="5adc898f-40d4-4851-af95-1f400d519360" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Oct 29 23:26:57.303219 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:57.303181 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-nbfzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Oct 29 23:26:57.303397 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:57.303251 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" podUID="5adc898f-40d4-4851-af95-1f400d519360" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Oct 29 23:26:58.303552 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:58.303514 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-nbfzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Oct 29 23:26:58.303964 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:58.303566 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" podUID="5adc898f-40d4-4851-af95-1f400d519360" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Oct 29 23:26:59.303095 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:59.303055 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-nbfzf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Oct 29 23:26:59.303285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:26:59.303121 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" podUID="5adc898f-40d4-4851-af95-1f400d519360" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Oct 29 23:27:00.930852 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:00.930816 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:27:00.931364 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:00.931237 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:27:00.931978 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:00.931957 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" Oct 29 23:27:22.395382 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:22.395315 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:22.415768 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:27:22.415684 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:27:22.559128 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:22.559092 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:22.937780 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:22.937749 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr"] Oct 29 23:27:22.946366 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:22.946339 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:22.952690 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:22.952653 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:22.966141 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:22.966109 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-92f6b3-openshift-default-dockercfg-lbv8z\"" Oct 29 23:27:22.968096 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:22.968073 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr"] Oct 29 23:27:22.990867 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:22.990768 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:22.991085 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:22.991063 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf4lr\" (UniqueName: \"kubernetes.io/projected/40c7dc3a-342c-48a9-a569-d9545b1a6653-kube-api-access-mf4lr\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:22.991251 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:22.991235 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:22.991372 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:22.991353 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:22.991455 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:22.991397 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:22.991455 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:22.991446 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:22.991562 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:22.991478 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c7dc3a-342c-48a9-a569-d9545b1a6653-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:22.991562 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:22.991524 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:22.991665 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:22.991571 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.092529 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.092495 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.092529 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.092543 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.092786 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.092568 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.092786 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.092599 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf4lr\" (UniqueName: \"kubernetes.io/projected/40c7dc3a-342c-48a9-a569-d9545b1a6653-kube-api-access-mf4lr\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.092786 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.092649 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.092786 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.092667 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.092786 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.092693 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.092786 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.092722 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.092786 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.092743 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c7dc3a-342c-48a9-a569-d9545b1a6653-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.093350 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.093322 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c7dc3a-342c-48a9-a569-d9545b1a6653-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.093964 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.093938 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.094109 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.093965 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.094208 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.093988 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.094318 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.094237 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.096395 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.096363 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.096517 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.096444 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.109745 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.109717 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz"] Oct 29 23:27:23.115236 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.115213 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.123496 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.123470 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.125074 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.125049 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf4lr\" (UniqueName: \"kubernetes.io/projected/40c7dc3a-342c-48a9-a569-d9545b1a6653-kube-api-access-mf4lr\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.139621 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.139172 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-2a53c9-openshift-default-dockercfg-8qsfl\"" Oct 29 23:27:23.143702 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.143200 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz"] Oct 29 23:27:23.147422 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.147395 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:23.193732 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.193654 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29e6194d-41e0-4e53-8698-32ace257af0e-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.193732 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.193711 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29e6194d-41e0-4e53-8698-32ace257af0e-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.193941 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.193779 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.193941 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.193807 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkpvv\" (UniqueName: \"kubernetes.io/projected/29e6194d-41e0-4e53-8698-32ace257af0e-kube-api-access-wkpvv\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.193941 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.193825 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.193941 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.193840 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.193941 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.193858 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.194189 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.194073 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29e6194d-41e0-4e53-8698-32ace257af0e-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.194189 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.194118 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.275353 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.275314 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:23.295637 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.295600 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29e6194d-41e0-4e53-8698-32ace257af0e-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.295767 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.295652 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.295767 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.295696 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkpvv\" (UniqueName: \"kubernetes.io/projected/29e6194d-41e0-4e53-8698-32ace257af0e-kube-api-access-wkpvv\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.295767 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.295722 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.295767 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.295749 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.296076 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.295775 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.296076 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.295898 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29e6194d-41e0-4e53-8698-32ace257af0e-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.296076 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.295928 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.296076 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.295961 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29e6194d-41e0-4e53-8698-32ace257af0e-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.298446 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.297289 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29e6194d-41e0-4e53-8698-32ace257af0e-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.298446 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.297591 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.298446 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.297854 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.298446 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.298368 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.299376 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.299349 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29e6194d-41e0-4e53-8698-32ace257af0e-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.299688 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.299670 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.301533 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.301465 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.325524 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.325416 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29e6194d-41e0-4e53-8698-32ace257af0e-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.331412 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.331349 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkpvv\" (UniqueName: \"kubernetes.io/projected/29e6194d-41e0-4e53-8698-32ace257af0e-kube-api-access-wkpvv\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.343954 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.343777 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:23.447064 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.446987 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:23.549936 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.549880 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:23.614521 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.614499 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr"] Oct 29 23:27:23.615722 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:27:23.615696 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c7dc3a_342c_48a9_a569_d9545b1a6653.slice/crio-b1d6597eb96bf712316f110f5b4b25d1d2e98478ce0960b143b841f7ca28eb49 WatchSource:0}: Error finding container b1d6597eb96bf712316f110f5b4b25d1d2e98478ce0960b143b841f7ca28eb49: Status 404 returned error can't find the container with id b1d6597eb96bf712316f110f5b4b25d1d2e98478ce0960b143b841f7ca28eb49 Oct 29 23:27:23.618030 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.617991 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:27:23.618155 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.618085 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:27:23.618155 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.618118 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:27:23.642180 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.642154 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g"] Oct 29 23:27:23.647212 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.647194 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.672476 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.672403 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g"] Oct 29 23:27:23.700246 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.700178 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7skr\" (UniqueName: \"kubernetes.io/projected/b46bbd7f-4b97-4b30-b500-1c31f0507898-kube-api-access-q7skr\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.700246 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.700233 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b46bbd7f-4b97-4b30-b500-1c31f0507898-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.700406 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.700348 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.700406 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.700384 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.700496 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.700412 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.700595 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.700578 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.700658 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.700612 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.700658 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.700639 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.700785 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.700765 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.719727 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.719704 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-ae1784-openshift-default-dockercfg-bf8gx\"" Oct 29 23:27:23.727690 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.727663 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:23.774148 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.774115 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz"] Oct 29 23:27:23.778497 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:27:23.778462 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29e6194d_41e0_4e53_8698_32ace257af0e.slice/crio-ca8f504f55b17b645aa60ea0fdc8654b105c86da58088775201c99eafba72433 WatchSource:0}: Error finding container ca8f504f55b17b645aa60ea0fdc8654b105c86da58088775201c99eafba72433: Status 404 returned error can't find the container with id ca8f504f55b17b645aa60ea0fdc8654b105c86da58088775201c99eafba72433 Oct 29 23:27:23.781349 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.781313 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:27:23.781452 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.781390 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:27:23.781524 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.781454 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:27:23.805139 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.805107 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7skr\" (UniqueName: \"kubernetes.io/projected/b46bbd7f-4b97-4b30-b500-1c31f0507898-kube-api-access-q7skr\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.808418 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.808394 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b46bbd7f-4b97-4b30-b500-1c31f0507898-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.808562 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.808548 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.808680 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.808667 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.808784 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.808768 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.808925 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.808913 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.809011 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.808999 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.809091 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.809079 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.809251 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.809223 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.809751 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.809720 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.810173 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.808616 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v"] Oct 29 23:27:23.810500 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.810482 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.810607 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.810538 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.810821 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.810767 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.812894 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.812866 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.815435 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.815403 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b46bbd7f-4b97-4b30-b500-1c31f0507898-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.815908 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.815892 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:23.817900 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.817816 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.824385 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.824108 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" event={"ID":"29e6194d-41e0-4e53-8698-32ace257af0e","Type":"ContainerStarted","Data":"ca8f504f55b17b645aa60ea0fdc8654b105c86da58088775201c99eafba72433"} Oct 29 23:27:23.825710 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.825687 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" event={"ID":"40c7dc3a-342c-48a9-a569-d9545b1a6653","Type":"ContainerStarted","Data":"bc858af7fef041166fdb7db0b7e1e4e6354bc8ddba738cc03a7606cfb0ee4eae"} Oct 29 23:27:23.825795 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.825729 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" event={"ID":"40c7dc3a-342c-48a9-a569-d9545b1a6653","Type":"ContainerStarted","Data":"b1d6597eb96bf712316f110f5b4b25d1d2e98478ce0960b143b841f7ca28eb49"} Oct 29 23:27:23.841627 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.841588 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-ae1364-openshift-default-dockercfg-9nkvr\"" Oct 29 23:27:23.849147 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.849115 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v"] Oct 29 23:27:23.850877 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.850824 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7skr\" (UniqueName: \"kubernetes.io/projected/b46bbd7f-4b97-4b30-b500-1c31f0507898-kube-api-access-q7skr\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.854298 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.854279 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:23.910708 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.910059 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:23.910708 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.910173 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plgnk\" (UniqueName: \"kubernetes.io/projected/1a24e9f0-98ce-4367-99b2-25653f0171ee-kube-api-access-plgnk\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:23.910708 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.910222 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:23.910708 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.910250 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:23.910708 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.910303 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:23.910708 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.910360 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a24e9f0-98ce-4367-99b2-25653f0171ee-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:23.910708 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.910413 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:23.910708 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.910459 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:23.910708 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.910525 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:23.914986 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.914942 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:23.953541 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.947118 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" podStartSLOduration=1.9470976690000001 podStartE2EDuration="1.947097669s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:23.934891851 +0000 UTC m=+871.301639482" watchObservedRunningTime="2026-10-29 23:27:23.947097669 +0000 UTC m=+871.313845301" Oct 29 23:27:23.988363 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:23.988326 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:24.011861 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.011827 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.012041 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.012025 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.012150 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.012137 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.012291 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.012277 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a24e9f0-98ce-4367-99b2-25653f0171ee-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.012438 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.012421 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.012542 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.012528 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.012650 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.012638 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.012771 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.012760 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.012920 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.012907 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plgnk\" (UniqueName: \"kubernetes.io/projected/1a24e9f0-98ce-4367-99b2-25653f0171ee-kube-api-access-plgnk\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.016450 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.013816 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.020209 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.019563 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.020209 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.019639 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.020209 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.019671 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.020209 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.020162 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a24e9f0-98ce-4367-99b2-25653f0171ee-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.022779 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.021002 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.024914 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.023103 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.045986 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.045953 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-plgnk\" (UniqueName: \"kubernetes.io/projected/1a24e9f0-98ce-4367-99b2-25653f0171ee-kube-api-access-plgnk\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.047670 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.047646 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.141019 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.140964 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:24.158010 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.157800 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:24.276713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.276685 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:24.281709 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.281682 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:24.336972 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.327554 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:27:24.336972 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.327697 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:27:24.336972 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.327805 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:27:24.336972 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.334554 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:24.336972 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.336600 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g"] Oct 29 23:27:24.474972 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.474947 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v"] Oct 29 23:27:24.476928 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:27:24.476898 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a24e9f0_98ce_4367_99b2_25653f0171ee.slice/crio-e780e2b42d5f611882fc5b47bd7ed061947048ce4e16dce7fb2e99ae0c92ce62 WatchSource:0}: Error finding container e780e2b42d5f611882fc5b47bd7ed061947048ce4e16dce7fb2e99ae0c92ce62: Status 404 returned error can't find the container with id e780e2b42d5f611882fc5b47bd7ed061947048ce4e16dce7fb2e99ae0c92ce62 Oct 29 23:27:24.479424 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.479389 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:27:24.479515 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.479461 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:27:24.479515 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.479491 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:27:24.835763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.835374 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" event={"ID":"b46bbd7f-4b97-4b30-b500-1c31f0507898","Type":"ContainerStarted","Data":"b684f331163d46ae3a985263979830c8fd0d38f46e2c5df184da198dd480426d"} Oct 29 23:27:24.843585 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.836136 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" event={"ID":"b46bbd7f-4b97-4b30-b500-1c31f0507898","Type":"ContainerStarted","Data":"70339fbf635eff23c76a09d889f1220de8fc440393a85c59404ef06a58ce0e15"} Oct 29 23:27:24.843585 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.837528 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" event={"ID":"1a24e9f0-98ce-4367-99b2-25653f0171ee","Type":"ContainerStarted","Data":"c7c965a4faa698e70a94442ca3e2686c47cb0c57651d2d94f414611b6cf3ca6b"} Oct 29 23:27:24.843585 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.837578 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" event={"ID":"1a24e9f0-98ce-4367-99b2-25653f0171ee","Type":"ContainerStarted","Data":"e780e2b42d5f611882fc5b47bd7ed061947048ce4e16dce7fb2e99ae0c92ce62"} Oct 29 23:27:24.843585 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.840052 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" event={"ID":"29e6194d-41e0-4e53-8698-32ace257af0e","Type":"ContainerStarted","Data":"4e2c81a74eb98b84f566faafb4f40f4148e0b7f4bb3c15b22a79f45d4998a2c1"} Oct 29 23:27:24.843585 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.840286 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:24.843585 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.841214 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:24.886540 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.886489 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" podStartSLOduration=1.886474956 podStartE2EDuration="1.886474956s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:24.885555729 +0000 UTC m=+872.252303361" watchObservedRunningTime="2026-10-29 23:27:24.886474956 +0000 UTC m=+872.253222587" Oct 29 23:27:24.934756 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.934706 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" podStartSLOduration=1.9346923550000001 podStartE2EDuration="1.934692355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:24.93304918 +0000 UTC m=+872.299796812" watchObservedRunningTime="2026-10-29 23:27:24.934692355 +0000 UTC m=+872.301439985" Oct 29 23:27:24.982462 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.982416 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" podStartSLOduration=1.982405166 podStartE2EDuration="1.982405166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:24.980974957 +0000 UTC m=+872.347722588" watchObservedRunningTime="2026-10-29 23:27:24.982405166 +0000 UTC m=+872.349152797" Oct 29 23:27:24.988661 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:24.988632 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:25.163298 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:25.163116 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:25.175396 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:25.175364 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:25.447666 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:25.447587 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:25.452341 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:25.452317 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:25.846964 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:25.846876 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:25.847450 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:25.847424 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:25.847554 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:25.847482 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:27:25.848368 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:25.848343 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:27:25.995477 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:25.995445 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:26.540858 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:26.540820 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:26.733192 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:26.733154 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:26.852959 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:26.852879 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:26.853747 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:26.853729 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:27:26.942926 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:26.942895 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:27.129751 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:27.129675 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:27.348018 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:27.347983 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:27.665641 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:27.665611 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:27.896273 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:27.896224 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:28.102411 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:28.102324 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:28.303586 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:28.303551 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:28.945048 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:28.945016 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:29.160283 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:29.156845 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:30.297614 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.297580 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb"] Oct 29 23:27:30.302578 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.302557 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:27:30.341363 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.341332 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb"] Oct 29 23:27:30.356294 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.356232 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-53c44e-serving-cert\"" Oct 29 23:27:30.356294 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.356279 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-k7tlz\"" Oct 29 23:27:30.383370 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.383336 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a2facd5-3558-46ca-a031-37c6874548a2-metrics-certs\") pod \"maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb\" (UID: \"9a2facd5-3558-46ca-a031-37c6874548a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:27:30.383539 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.383423 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-628sz\" (UniqueName: \"kubernetes.io/projected/9a2facd5-3558-46ca-a031-37c6874548a2-kube-api-access-628sz\") pod \"maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb\" (UID: \"9a2facd5-3558-46ca-a031-37c6874548a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:27:30.383539 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.383475 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a2facd5-3558-46ca-a031-37c6874548a2-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb\" (UID: \"9a2facd5-3558-46ca-a031-37c6874548a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:27:30.383774 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.383620 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a2facd5-3558-46ca-a031-37c6874548a2-maas-api-tls\") pod \"maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb\" (UID: \"9a2facd5-3558-46ca-a031-37c6874548a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:27:30.468783 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.468751 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:30.484877 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.484841 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a2facd5-3558-46ca-a031-37c6874548a2-metrics-certs\") pod \"maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb\" (UID: \"9a2facd5-3558-46ca-a031-37c6874548a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:27:30.485064 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.484916 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-628sz\" (UniqueName: \"kubernetes.io/projected/9a2facd5-3558-46ca-a031-37c6874548a2-kube-api-access-628sz\") pod \"maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb\" (UID: \"9a2facd5-3558-46ca-a031-37c6874548a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:27:30.485064 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.484959 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a2facd5-3558-46ca-a031-37c6874548a2-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb\" (UID: \"9a2facd5-3558-46ca-a031-37c6874548a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:27:30.485064 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.485022 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a2facd5-3558-46ca-a031-37c6874548a2-maas-api-tls\") pod \"maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb\" (UID: \"9a2facd5-3558-46ca-a031-37c6874548a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:27:30.485905 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.485773 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a2facd5-3558-46ca-a031-37c6874548a2-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb\" (UID: \"9a2facd5-3558-46ca-a031-37c6874548a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:27:30.487664 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.487642 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a2facd5-3558-46ca-a031-37c6874548a2-metrics-certs\") pod \"maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb\" (UID: \"9a2facd5-3558-46ca-a031-37c6874548a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:27:30.487821 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.487801 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a2facd5-3558-46ca-a031-37c6874548a2-maas-api-tls\") pod \"maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb\" (UID: \"9a2facd5-3558-46ca-a031-37c6874548a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:27:30.510284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.510234 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-628sz\" (UniqueName: \"kubernetes.io/projected/9a2facd5-3558-46ca-a031-37c6874548a2-kube-api-access-628sz\") pod \"maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb\" (UID: \"9a2facd5-3558-46ca-a031-37c6874548a2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:27:30.615886 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.615767 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:27:30.657716 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.657687 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:30.923217 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.923188 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:30.936532 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:30.936508 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb"] Oct 29 23:27:30.938620 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:27:30.938598 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a2facd5_3558_46ca_a031_37c6874548a2.slice/crio-6d6048a541fcdd895eca1749630946ae9734e212bb9b2f7ed463b8edd7d4c975 WatchSource:0}: Error finding container 6d6048a541fcdd895eca1749630946ae9734e212bb9b2f7ed463b8edd7d4c975: Status 404 returned error can't find the container with id 6d6048a541fcdd895eca1749630946ae9734e212bb9b2f7ed463b8edd7d4c975 Oct 29 23:27:31.633037 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:31.632998 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:31.814791 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:31.814668 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:31.901082 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:31.901042 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" event={"ID":"9a2facd5-3558-46ca-a031-37c6874548a2","Type":"ContainerStarted","Data":"8afd2854f5f8f4fc20d6a4d9d7e61e267c8d14bee8e821ee897fd3b079332691"} Oct 29 23:27:31.901316 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:31.901109 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" event={"ID":"9a2facd5-3558-46ca-a031-37c6874548a2","Type":"ContainerStarted","Data":"6d6048a541fcdd895eca1749630946ae9734e212bb9b2f7ed463b8edd7d4c975"} Oct 29 23:27:31.902353 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:31.902067 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:27:31.949505 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:31.949447 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" podStartSLOduration=1.369661002 podStartE2EDuration="1.94943404s" totalImagesPullingTime="579.773038ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:31.948890744 +0000 UTC m=+879.315638376" watchObservedRunningTime="2026-10-29 23:27:31.94943404 +0000 UTC m=+879.316181671" Oct 29 23:27:32.442321 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.442256 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:32.645162 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.645102 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:32.694343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.693570 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs"] Oct 29 23:27:32.705968 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.699853 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:27:32.716622 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.716593 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs"] Oct 29 23:27:32.725332 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.725256 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-b978a0-serving-cert\"" Oct 29 23:27:32.806474 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.806440 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:32.816739 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.816705 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c8f079-e7ec-4d31-9477-e9edc61446db-metrics-certs\") pod \"maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs\" (UID: \"d3c8f079-e7ec-4d31-9477-e9edc61446db\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:27:32.816882 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.816801 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c8f079-e7ec-4d31-9477-e9edc61446db-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs\" (UID: \"d3c8f079-e7ec-4d31-9477-e9edc61446db\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:27:32.816882 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.816833 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnhch\" (UniqueName: \"kubernetes.io/projected/d3c8f079-e7ec-4d31-9477-e9edc61446db-kube-api-access-jnhch\") pod \"maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs\" (UID: \"d3c8f079-e7ec-4d31-9477-e9edc61446db\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:27:32.816983 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.816937 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3c8f079-e7ec-4d31-9477-e9edc61446db-maas-api-tls\") pod \"maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs\" (UID: \"d3c8f079-e7ec-4d31-9477-e9edc61446db\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:27:32.918468 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.918430 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c8f079-e7ec-4d31-9477-e9edc61446db-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs\" (UID: \"d3c8f079-e7ec-4d31-9477-e9edc61446db\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:27:32.918656 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.918501 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnhch\" (UniqueName: \"kubernetes.io/projected/d3c8f079-e7ec-4d31-9477-e9edc61446db-kube-api-access-jnhch\") pod \"maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs\" (UID: \"d3c8f079-e7ec-4d31-9477-e9edc61446db\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:27:32.918656 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.918602 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3c8f079-e7ec-4d31-9477-e9edc61446db-maas-api-tls\") pod \"maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs\" (UID: \"d3c8f079-e7ec-4d31-9477-e9edc61446db\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:27:32.918769 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.918740 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c8f079-e7ec-4d31-9477-e9edc61446db-metrics-certs\") pod \"maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs\" (UID: \"d3c8f079-e7ec-4d31-9477-e9edc61446db\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:27:32.919040 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.919014 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c8f079-e7ec-4d31-9477-e9edc61446db-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs\" (UID: \"d3c8f079-e7ec-4d31-9477-e9edc61446db\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:27:32.921120 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.921093 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3c8f079-e7ec-4d31-9477-e9edc61446db-maas-api-tls\") pod \"maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs\" (UID: \"d3c8f079-e7ec-4d31-9477-e9edc61446db\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:27:32.921209 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.921158 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c8f079-e7ec-4d31-9477-e9edc61446db-metrics-certs\") pod \"maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs\" (UID: \"d3c8f079-e7ec-4d31-9477-e9edc61446db\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:27:32.943485 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.943455 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnhch\" (UniqueName: \"kubernetes.io/projected/d3c8f079-e7ec-4d31-9477-e9edc61446db-kube-api-access-jnhch\") pod \"maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs\" (UID: \"d3c8f079-e7ec-4d31-9477-e9edc61446db\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:27:32.962768 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:32.962695 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:33.030224 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:33.030195 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:27:33.134768 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:33.134731 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:33.332500 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:33.332471 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs"] Oct 29 23:27:33.333743 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:27:33.333717 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3c8f079_e7ec_4d31_9477_e9edc61446db.slice/crio-e7904a2616fc3ce511e6cf5d02c4542018cdb12fc0dccd418226b7e4c5114d35 WatchSource:0}: Error finding container e7904a2616fc3ce511e6cf5d02c4542018cdb12fc0dccd418226b7e4c5114d35: Status 404 returned error can't find the container with id e7904a2616fc3ce511e6cf5d02c4542018cdb12fc0dccd418226b7e4c5114d35 Oct 29 23:27:33.914635 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:33.914545 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" event={"ID":"d3c8f079-e7ec-4d31-9477-e9edc61446db","Type":"ContainerStarted","Data":"4aacefbad2fcfafed218ea646dce6127541da2e325588ed467ecfb0d44bacc44"} Oct 29 23:27:33.914635 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:33.914610 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" event={"ID":"d3c8f079-e7ec-4d31-9477-e9edc61446db","Type":"ContainerStarted","Data":"e7904a2616fc3ce511e6cf5d02c4542018cdb12fc0dccd418226b7e4c5114d35"} Oct 29 23:27:33.915029 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:33.914791 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:27:33.961665 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:33.961607 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" podStartSLOduration=1.637031911 podStartE2EDuration="1.961591193s" totalImagesPullingTime="324.559282ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:33.958973923 +0000 UTC m=+881.325721646" watchObservedRunningTime="2026-10-29 23:27:33.961591193 +0000 UTC m=+881.328338823" Oct 29 23:27:34.072096 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.072059 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:34.196546 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.196467 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk"] Oct 29 23:27:34.200389 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.200365 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" Oct 29 23:27:34.219546 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.219525 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-2a53c9-dockercfg-9dw7c\"" Oct 29 23:27:34.222241 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.222219 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk"] Oct 29 23:27:34.245716 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.245653 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:34.253062 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.253032 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-2a53c9\"" Oct 29 23:27:34.333290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.333233 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h5qm\" (UniqueName: \"kubernetes.io/projected/4cb6d795-205b-465f-91b0-fa184fa8fe44-kube-api-access-6h5qm\") pod \"payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk\" (UID: \"4cb6d795-205b-465f-91b0-fa184fa8fe44\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" Oct 29 23:27:34.333483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.333436 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cb6d795-205b-465f-91b0-fa184fa8fe44-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk\" (UID: \"4cb6d795-205b-465f-91b0-fa184fa8fe44\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" Oct 29 23:27:34.437283 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.436725 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cb6d795-205b-465f-91b0-fa184fa8fe44-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk\" (UID: \"4cb6d795-205b-465f-91b0-fa184fa8fe44\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" Oct 29 23:27:34.437283 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.436814 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h5qm\" (UniqueName: \"kubernetes.io/projected/4cb6d795-205b-465f-91b0-fa184fa8fe44-kube-api-access-6h5qm\") pod \"payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk\" (UID: \"4cb6d795-205b-465f-91b0-fa184fa8fe44\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" Oct 29 23:27:34.441280 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.438052 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cb6d795-205b-465f-91b0-fa184fa8fe44-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk\" (UID: \"4cb6d795-205b-465f-91b0-fa184fa8fe44\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" Oct 29 23:27:34.472489 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.472406 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h5qm\" (UniqueName: \"kubernetes.io/projected/4cb6d795-205b-465f-91b0-fa184fa8fe44-kube-api-access-6h5qm\") pod \"payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk\" (UID: \"4cb6d795-205b-465f-91b0-fa184fa8fe44\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" Oct 29 23:27:34.514394 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.514360 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" Oct 29 23:27:34.600873 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.600829 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48"] Oct 29 23:27:34.607942 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.607910 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:27:34.628365 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.628335 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-92f6b3-serving-cert\"" Oct 29 23:27:34.630841 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.630814 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48"] Oct 29 23:27:34.740158 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.740065 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-547fv\" (UniqueName: \"kubernetes.io/projected/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-kube-api-access-547fv\") pod \"maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48\" (UID: \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:27:34.740346 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.740208 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-maas-api-tls\") pod \"maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48\" (UID: \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:27:34.740412 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.740357 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48\" (UID: \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:27:34.740462 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.740411 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-metrics-certs\") pod \"maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48\" (UID: \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:27:34.752353 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.752326 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:34.840370 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.840324 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk"] Oct 29 23:27:34.863292 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.859982 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48\" (UID: \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:27:34.863292 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.860050 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-metrics-certs\") pod \"maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48\" (UID: \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:27:34.863292 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.860093 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-547fv\" (UniqueName: \"kubernetes.io/projected/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-kube-api-access-547fv\") pod \"maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48\" (UID: \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:27:34.863292 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.860179 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-maas-api-tls\") pod \"maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48\" (UID: \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:27:34.863292 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:27:34.860402 5056 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-serving-cert: secret "maas-api-e2e-worker-w3-92f6b3-serving-cert" not found Oct 29 23:27:34.863292 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:27:34.860554 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-maas-api-tls podName:6d57ed6a-5cef-46d3-8449-2eefc9e26c20 nodeName:}" failed. No retries permitted until 2026-10-29 23:27:35.360448075 +0000 UTC m=+882.727195708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-maas-api-tls") pod "maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" (UID: "6d57ed6a-5cef-46d3-8449-2eefc9e26c20") : secret "maas-api-e2e-worker-w3-92f6b3-serving-cert" not found Oct 29 23:27:34.863292 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.861469 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48\" (UID: \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:27:34.867951 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.867921 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-metrics-certs\") pod \"maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48\" (UID: \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:27:34.903906 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.903877 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-547fv\" (UniqueName: \"kubernetes.io/projected/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-kube-api-access-547fv\") pod \"maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48\" (UID: \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:27:34.921609 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.921582 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" event={"ID":"4cb6d795-205b-465f-91b0-fa184fa8fe44","Type":"ContainerStarted","Data":"9238524f85f6c9bcf44313444f7ae6bb5bbbc0178cdf3582d56bad1fe2f41199"} Oct 29 23:27:34.987554 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:34.987512 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:35.146394 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:35.146361 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:35.366497 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:35.366457 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-maas-api-tls\") pod \"maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48\" (UID: \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:27:35.369255 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:35.369223 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-maas-api-tls\") pod \"maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48\" (UID: \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:27:35.524806 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:35.524718 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:27:35.760596 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:35.760562 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:35.853746 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:35.853717 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48"] Oct 29 23:27:35.854620 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:27:35.854591 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d57ed6a_5cef_46d3_8449_2eefc9e26c20.slice/crio-7cefb2468cfeb291cc26b691efb65611d9accdb01f3800b4cd7fa6ba36a84863 WatchSource:0}: Error finding container 7cefb2468cfeb291cc26b691efb65611d9accdb01f3800b4cd7fa6ba36a84863: Status 404 returned error can't find the container with id 7cefb2468cfeb291cc26b691efb65611d9accdb01f3800b4cd7fa6ba36a84863 Oct 29 23:27:35.928768 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:35.928736 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:35.929228 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:35.928830 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" event={"ID":"6d57ed6a-5cef-46d3-8449-2eefc9e26c20","Type":"ContainerStarted","Data":"7cefb2468cfeb291cc26b691efb65611d9accdb01f3800b4cd7fa6ba36a84863"} Oct 29 23:27:35.930826 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:35.930790 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" event={"ID":"4cb6d795-205b-465f-91b0-fa184fa8fe44","Type":"ContainerStarted","Data":"f9e75e3916a6ba4fcd6426fd98138674c80f81b5bdf4aad07ac2e244de5d0154"} Oct 29 23:27:35.930962 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:35.930922 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" Oct 29 23:27:35.977357 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:35.977304 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" podStartSLOduration=1.977292114 podStartE2EDuration="1.977292114s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:35.976654876 +0000 UTC m=+883.343402507" watchObservedRunningTime="2026-10-29 23:27:35.977292114 +0000 UTC m=+883.344039745" Oct 29 23:27:36.760768 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:36.760732 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:36.937538 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:36.937505 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" event={"ID":"6d57ed6a-5cef-46d3-8449-2eefc9e26c20","Type":"ContainerStarted","Data":"92be338c57782b8c60020785fe4ff4c740f506470c7c42eac0536dff68d72673"} Oct 29 23:27:36.984470 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:36.984401 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" podStartSLOduration=2.667542729 podStartE2EDuration="2.984381843s" totalImagesPullingTime="316.839114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:36.98429904 +0000 UTC m=+884.351046669" watchObservedRunningTime="2026-10-29 23:27:36.984381843 +0000 UTC m=+884.351129475" Oct 29 23:27:37.334371 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:37.334338 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:37.506281 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:37.505455 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:37.942793 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:37.942763 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:27:38.547385 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.547347 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk"] Oct 29 23:27:38.552051 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.552028 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:27:38.583757 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.583726 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-aa7e96-serving-cert\"" Oct 29 23:27:38.587382 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.587352 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk"] Oct 29 23:27:38.678112 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.678080 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:38.705297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.705274 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76b760f1-81f7-460a-bb76-1319cd789c50-maas-api-tls\") pod \"maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk\" (UID: \"76b760f1-81f7-460a-bb76-1319cd789c50\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:27:38.705460 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.705425 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/76b760f1-81f7-460a-bb76-1319cd789c50-metrics-certs\") pod \"maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk\" (UID: \"76b760f1-81f7-460a-bb76-1319cd789c50\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:27:38.705527 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.705474 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76b760f1-81f7-460a-bb76-1319cd789c50-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk\" (UID: \"76b760f1-81f7-460a-bb76-1319cd789c50\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:27:38.705591 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.705576 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2jzz\" (UniqueName: \"kubernetes.io/projected/76b760f1-81f7-460a-bb76-1319cd789c50-kube-api-access-j2jzz\") pod \"maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk\" (UID: \"76b760f1-81f7-460a-bb76-1319cd789c50\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:27:38.811750 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.811657 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2jzz\" (UniqueName: \"kubernetes.io/projected/76b760f1-81f7-460a-bb76-1319cd789c50-kube-api-access-j2jzz\") pod \"maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk\" (UID: \"76b760f1-81f7-460a-bb76-1319cd789c50\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:27:38.812971 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.812942 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76b760f1-81f7-460a-bb76-1319cd789c50-maas-api-tls\") pod \"maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk\" (UID: \"76b760f1-81f7-460a-bb76-1319cd789c50\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:27:38.813326 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.813304 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/76b760f1-81f7-460a-bb76-1319cd789c50-metrics-certs\") pod \"maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk\" (UID: \"76b760f1-81f7-460a-bb76-1319cd789c50\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:27:38.813624 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.813605 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76b760f1-81f7-460a-bb76-1319cd789c50-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk\" (UID: \"76b760f1-81f7-460a-bb76-1319cd789c50\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:27:38.814750 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.814726 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76b760f1-81f7-460a-bb76-1319cd789c50-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk\" (UID: \"76b760f1-81f7-460a-bb76-1319cd789c50\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:27:38.825099 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.825042 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/76b760f1-81f7-460a-bb76-1319cd789c50-metrics-certs\") pod \"maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk\" (UID: \"76b760f1-81f7-460a-bb76-1319cd789c50\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:27:38.825856 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.825828 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76b760f1-81f7-460a-bb76-1319cd789c50-maas-api-tls\") pod \"maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk\" (UID: \"76b760f1-81f7-460a-bb76-1319cd789c50\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:27:38.836410 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.836388 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:38.837143 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.837121 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2jzz\" (UniqueName: \"kubernetes.io/projected/76b760f1-81f7-460a-bb76-1319cd789c50-kube-api-access-j2jzz\") pod \"maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk\" (UID: \"76b760f1-81f7-460a-bb76-1319cd789c50\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:27:38.869313 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.869292 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:27:38.922365 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:38.921959 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:27:39.173313 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:39.173283 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk"] Oct 29 23:27:39.175121 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:27:39.175095 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76b760f1_81f7_460a_bb76_1319cd789c50.slice/crio-d90e59ece49b2901dfe9165545948412a34d4168cced2b578f42f73a33685e97 WatchSource:0}: Error finding container d90e59ece49b2901dfe9165545948412a34d4168cced2b578f42f73a33685e97: Status 404 returned error can't find the container with id d90e59ece49b2901dfe9165545948412a34d4168cced2b578f42f73a33685e97 Oct 29 23:27:39.935927 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:39.934689 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:27:39.963651 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:39.963612 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" event={"ID":"76b760f1-81f7-460a-bb76-1319cd789c50","Type":"ContainerStarted","Data":"f8b7cfe3f9e17d2bce35afa6cffc12937971b8dceaefabf115bee36bc9fad204"} Oct 29 23:27:39.963837 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:39.963678 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:27:39.963837 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:39.963700 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" event={"ID":"76b760f1-81f7-460a-bb76-1319cd789c50","Type":"ContainerStarted","Data":"d90e59ece49b2901dfe9165545948412a34d4168cced2b578f42f73a33685e97"} Oct 29 23:27:40.435285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.435015 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" podStartSLOduration=2.110731662 podStartE2EDuration="2.434994942s" totalImagesPullingTime="324.26328ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:40.041382452 +0000 UTC m=+887.408130084" watchObservedRunningTime="2026-10-29 23:27:40.434994942 +0000 UTC m=+887.801742574" Oct 29 23:27:40.443338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.442300 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7"] Oct 29 23:27:40.447698 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.447673 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:27:40.465848 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.465815 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-2a53c9-serving-cert\"" Oct 29 23:27:40.466041 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.466021 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7"] Oct 29 23:27:40.555286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.551438 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c103618d-ee4d-4568-bdaf-6e1b486f9277-maas-api-tls\") pod \"maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7\" (UID: \"c103618d-ee4d-4568-bdaf-6e1b486f9277\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:27:40.558602 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.551488 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c103618d-ee4d-4568-bdaf-6e1b486f9277-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7\" (UID: \"c103618d-ee4d-4568-bdaf-6e1b486f9277\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:27:40.558773 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.558734 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv2dj\" (UniqueName: \"kubernetes.io/projected/c103618d-ee4d-4568-bdaf-6e1b486f9277-kube-api-access-mv2dj\") pod \"maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7\" (UID: \"c103618d-ee4d-4568-bdaf-6e1b486f9277\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:27:40.558832 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.558801 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c103618d-ee4d-4568-bdaf-6e1b486f9277-metrics-certs\") pod \"maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7\" (UID: \"c103618d-ee4d-4568-bdaf-6e1b486f9277\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:27:40.595504 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.595470 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:40.664180 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.664137 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c103618d-ee4d-4568-bdaf-6e1b486f9277-maas-api-tls\") pod \"maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7\" (UID: \"c103618d-ee4d-4568-bdaf-6e1b486f9277\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:27:40.664432 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.664411 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c103618d-ee4d-4568-bdaf-6e1b486f9277-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7\" (UID: \"c103618d-ee4d-4568-bdaf-6e1b486f9277\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:27:40.664616 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.664603 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv2dj\" (UniqueName: \"kubernetes.io/projected/c103618d-ee4d-4568-bdaf-6e1b486f9277-kube-api-access-mv2dj\") pod \"maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7\" (UID: \"c103618d-ee4d-4568-bdaf-6e1b486f9277\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:27:40.664741 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.664728 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c103618d-ee4d-4568-bdaf-6e1b486f9277-metrics-certs\") pod \"maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7\" (UID: \"c103618d-ee4d-4568-bdaf-6e1b486f9277\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:27:40.668583 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.666098 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c103618d-ee4d-4568-bdaf-6e1b486f9277-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7\" (UID: \"c103618d-ee4d-4568-bdaf-6e1b486f9277\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:27:40.668767 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:27:40.668051 5056 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-serving-cert: secret "maas-api-e2e-worker-w1-2a53c9-serving-cert" not found Oct 29 23:27:40.672286 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:27:40.668933 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c103618d-ee4d-4568-bdaf-6e1b486f9277-maas-api-tls podName:c103618d-ee4d-4568-bdaf-6e1b486f9277 nodeName:}" failed. No retries permitted until 2026-10-29 23:27:41.168909886 +0000 UTC m=+888.535657516 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c103618d-ee4d-4568-bdaf-6e1b486f9277-maas-api-tls") pod "maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" (UID: "c103618d-ee4d-4568-bdaf-6e1b486f9277") : secret "maas-api-e2e-worker-w1-2a53c9-serving-cert" not found Oct 29 23:27:40.677565 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.677537 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c103618d-ee4d-4568-bdaf-6e1b486f9277-metrics-certs\") pod \"maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7\" (UID: \"c103618d-ee4d-4568-bdaf-6e1b486f9277\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:27:40.693252 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.693178 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv2dj\" (UniqueName: \"kubernetes.io/projected/c103618d-ee4d-4568-bdaf-6e1b486f9277-kube-api-access-mv2dj\") pod \"maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7\" (UID: \"c103618d-ee4d-4568-bdaf-6e1b486f9277\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:27:40.758014 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.757976 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:40.917285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:40.914382 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:41.169848 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:41.169809 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c103618d-ee4d-4568-bdaf-6e1b486f9277-maas-api-tls\") pod \"maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7\" (UID: \"c103618d-ee4d-4568-bdaf-6e1b486f9277\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:27:41.172180 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:41.172152 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c103618d-ee4d-4568-bdaf-6e1b486f9277-maas-api-tls\") pod \"maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7\" (UID: \"c103618d-ee4d-4568-bdaf-6e1b486f9277\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:27:41.372823 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:41.372785 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:27:41.705574 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:41.705546 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7"] Oct 29 23:27:41.706842 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:27:41.706818 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc103618d_ee4d_4568_bdaf_6e1b486f9277.slice/crio-00b50f0dca3da3107e050b65341a98676b2c2b652ac09ee4145fc97050000aa4 WatchSource:0}: Error finding container 00b50f0dca3da3107e050b65341a98676b2c2b652ac09ee4145fc97050000aa4: Status 404 returned error can't find the container with id 00b50f0dca3da3107e050b65341a98676b2c2b652ac09ee4145fc97050000aa4 Oct 29 23:27:41.976860 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:41.976767 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" event={"ID":"c103618d-ee4d-4568-bdaf-6e1b486f9277","Type":"ContainerStarted","Data":"00b50f0dca3da3107e050b65341a98676b2c2b652ac09ee4145fc97050000aa4"} Oct 29 23:27:42.524299 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:42.523908 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:42.983777 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:42.983742 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" event={"ID":"c103618d-ee4d-4568-bdaf-6e1b486f9277","Type":"ContainerStarted","Data":"b5102643cb97930058c6eae715e9aa7640f2a3d241242e73e1421f438e834c79"} Oct 29 23:27:42.984170 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:42.983834 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:27:43.035470 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:43.035410 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" podStartSLOduration=2.663057217 podStartE2EDuration="3.035390963s" totalImagesPullingTime="372.333746ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:43.034715983 +0000 UTC m=+890.401463625" watchObservedRunningTime="2026-10-29 23:27:43.035390963 +0000 UTC m=+890.402138594" Oct 29 23:27:43.956019 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:43.955991 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:27:44.156335 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:44.155977 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:44.337343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:44.336555 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:44.545903 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:44.545838 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:44.708557 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:44.708521 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:45.978286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:45.978244 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:27:46.680095 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.680063 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql"] Oct 29 23:27:46.684589 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.684566 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:27:46.702972 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.702942 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-ae1784-serving-cert\"" Oct 29 23:27:46.768311 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.764315 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql"] Oct 29 23:27:46.775401 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.774992 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e80bef80-0024-4113-b7f5-f43eded6d18e-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql\" (UID: \"e80bef80-0024-4113-b7f5-f43eded6d18e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:27:46.775401 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.775082 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e80bef80-0024-4113-b7f5-f43eded6d18e-metrics-certs\") pod \"maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql\" (UID: \"e80bef80-0024-4113-b7f5-f43eded6d18e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:27:46.775401 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.775116 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e80bef80-0024-4113-b7f5-f43eded6d18e-maas-api-tls\") pod \"maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql\" (UID: \"e80bef80-0024-4113-b7f5-f43eded6d18e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:27:46.775401 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.775162 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jttl\" (UniqueName: \"kubernetes.io/projected/e80bef80-0024-4113-b7f5-f43eded6d18e-kube-api-access-4jttl\") pod \"maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql\" (UID: \"e80bef80-0024-4113-b7f5-f43eded6d18e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:27:46.860286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.840161 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:46.876285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.876176 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e80bef80-0024-4113-b7f5-f43eded6d18e-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql\" (UID: \"e80bef80-0024-4113-b7f5-f43eded6d18e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:27:46.876285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.876254 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e80bef80-0024-4113-b7f5-f43eded6d18e-metrics-certs\") pod \"maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql\" (UID: \"e80bef80-0024-4113-b7f5-f43eded6d18e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:27:46.876285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.876292 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e80bef80-0024-4113-b7f5-f43eded6d18e-maas-api-tls\") pod \"maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql\" (UID: \"e80bef80-0024-4113-b7f5-f43eded6d18e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:27:46.876583 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.876323 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jttl\" (UniqueName: \"kubernetes.io/projected/e80bef80-0024-4113-b7f5-f43eded6d18e-kube-api-access-4jttl\") pod \"maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql\" (UID: \"e80bef80-0024-4113-b7f5-f43eded6d18e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:27:46.877240 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:27:46.877213 5056 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-serving-cert: secret "maas-api-e2e-worker-w6-ae1784-serving-cert" not found Oct 29 23:27:46.877395 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:27:46.877293 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e80bef80-0024-4113-b7f5-f43eded6d18e-maas-api-tls podName:e80bef80-0024-4113-b7f5-f43eded6d18e nodeName:}" failed. No retries permitted until 2026-10-29 23:27:47.377276687 +0000 UTC m=+894.744024315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e80bef80-0024-4113-b7f5-f43eded6d18e-maas-api-tls") pod "maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" (UID: "e80bef80-0024-4113-b7f5-f43eded6d18e") : secret "maas-api-e2e-worker-w6-ae1784-serving-cert" not found Oct 29 23:27:46.877539 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.877521 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e80bef80-0024-4113-b7f5-f43eded6d18e-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql\" (UID: \"e80bef80-0024-4113-b7f5-f43eded6d18e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:27:46.879180 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.879159 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e80bef80-0024-4113-b7f5-f43eded6d18e-metrics-certs\") pod \"maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql\" (UID: \"e80bef80-0024-4113-b7f5-f43eded6d18e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:27:46.900783 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.900749 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jttl\" (UniqueName: \"kubernetes.io/projected/e80bef80-0024-4113-b7f5-f43eded6d18e-kube-api-access-4jttl\") pod \"maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql\" (UID: \"e80bef80-0024-4113-b7f5-f43eded6d18e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:27:46.958952 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.958874 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" Oct 29 23:27:46.993203 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:46.993160 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:47.165298 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:47.165172 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:47.382447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:47.382359 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e80bef80-0024-4113-b7f5-f43eded6d18e-maas-api-tls\") pod \"maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql\" (UID: \"e80bef80-0024-4113-b7f5-f43eded6d18e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:27:47.387498 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:47.387470 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e80bef80-0024-4113-b7f5-f43eded6d18e-maas-api-tls\") pod \"maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql\" (UID: \"e80bef80-0024-4113-b7f5-f43eded6d18e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:27:47.539291 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:47.538774 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:47.599016 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:47.598976 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:27:47.713854 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:47.713782 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:48.043951 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:48.043916 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql"] Oct 29 23:27:48.045363 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:27:48.045334 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode80bef80_0024_4113_b7f5_f43eded6d18e.slice/crio-416cc7d2858b1be7fd5faf4f707d7568b8cb4150c8d2cf660a6281ac92c1a97f WatchSource:0}: Error finding container 416cc7d2858b1be7fd5faf4f707d7568b8cb4150c8d2cf660a6281ac92c1a97f: Status 404 returned error can't find the container with id 416cc7d2858b1be7fd5faf4f707d7568b8cb4150c8d2cf660a6281ac92c1a97f Oct 29 23:27:48.996924 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:48.996890 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:27:49.052286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:49.051980 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" event={"ID":"e80bef80-0024-4113-b7f5-f43eded6d18e","Type":"ContainerStarted","Data":"50637988903da9eee37952b4da1fbccdae355252ceb4ea02e87434770f457c03"} Oct 29 23:27:49.052286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:49.052049 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" event={"ID":"e80bef80-0024-4113-b7f5-f43eded6d18e","Type":"ContainerStarted","Data":"416cc7d2858b1be7fd5faf4f707d7568b8cb4150c8d2cf660a6281ac92c1a97f"} Oct 29 23:27:49.061312 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:49.052852 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:27:49.105992 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:49.105928 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" podStartSLOduration=2.6838207 podStartE2EDuration="3.105908412s" totalImagesPullingTime="422.087712ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:27:49.099733811 +0000 UTC m=+896.466481457" watchObservedRunningTime="2026-10-29 23:27:49.105908412 +0000 UTC m=+896.472656044" Oct 29 23:27:49.331345 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:49.317006 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:49.571407 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:49.571373 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:51.065643 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:51.065566 5056 generic.go:356] "Generic (PLEG): container finished" podID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerID="bc858af7fef041166fdb7db0b7e1e4e6354bc8ddba738cc03a7606cfb0ee4eae" exitCode=0 Oct 29 23:27:51.066058 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:51.065641 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" event={"ID":"40c7dc3a-342c-48a9-a569-d9545b1a6653","Type":"ContainerDied","Data":"bc858af7fef041166fdb7db0b7e1e4e6354bc8ddba738cc03a7606cfb0ee4eae"} Oct 29 23:27:51.066149 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:51.066134 5056 scope.go:122] "RemoveContainer" containerID="bc858af7fef041166fdb7db0b7e1e4e6354bc8ddba738cc03a7606cfb0ee4eae" Oct 29 23:27:51.067577 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:51.067552 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:27:51.067655 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:51.067615 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:27:51.067655 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:51.067646 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:27:51.844285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:51.844121 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:52.081630 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:52.081199 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" event={"ID":"40c7dc3a-342c-48a9-a569-d9545b1a6653","Type":"ContainerStarted","Data":"4e35ad5e6065f0cd661c8dde35970452e12b8de076e52cba24ec6a2d700282df"} Oct 29 23:27:52.276015 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:52.275969 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:52.277126 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:52.277101 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Oct 29 23:27:52.277243 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:52.277148 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" podUID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Oct 29 23:27:52.459005 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:27:52.458948 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:27:52.902752 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:52.902725 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:27:52.904055 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:52.904034 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:27:53.280535 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:53.276723 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Oct 29 23:27:53.280535 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:53.276785 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" podUID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Oct 29 23:27:53.940506 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:53.940422 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:54.276750 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:54.276657 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Oct 29 23:27:54.276750 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:54.276721 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" podUID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Oct 29 23:27:54.841106 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:54.841077 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:55.278767 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:55.278731 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Oct 29 23:27:55.279067 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:55.279037 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" podUID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Oct 29 23:27:55.648291 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:55.644664 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:55.948244 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:55.948141 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:56.107285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:56.092443 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:27:56.357500 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:56.357408 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Oct 29 23:27:56.357754 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:56.357724 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" podUID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Oct 29 23:27:56.607928 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:56.607788 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:56.796034 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:56.788188 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:56.890356 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:56.889398 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:27:56.890356 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:56.889485 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" podUID="29e6194d-41e0-4e53-8698-32ace257af0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:27:57.183249 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:57.183070 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:57.277287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:57.276706 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Oct 29 23:27:57.277287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:57.276769 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" podUID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Oct 29 23:27:57.390545 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:57.386600 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:57.613608 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:57.613387 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:57.852296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:57.852237 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:58.057385 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:58.057354 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:27:58.282721 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:58.282390 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:58.283593 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:58.283571 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:27:59.312598 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:27:59.312558 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:00.906962 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:00.906927 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:01.209324 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:01.209289 5056 generic.go:356] "Generic (PLEG): container finished" podID="29e6194d-41e0-4e53-8698-32ace257af0e" containerID="4e2c81a74eb98b84f566faafb4f40f4148e0b7f4bb3c15b22a79f45d4998a2c1" exitCode=0 Oct 29 23:28:01.209623 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:01.209310 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" event={"ID":"29e6194d-41e0-4e53-8698-32ace257af0e","Type":"ContainerDied","Data":"4e2c81a74eb98b84f566faafb4f40f4148e0b7f4bb3c15b22a79f45d4998a2c1"} Oct 29 23:28:01.210314 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:01.210293 5056 scope.go:122] "RemoveContainer" containerID="4e2c81a74eb98b84f566faafb4f40f4148e0b7f4bb3c15b22a79f45d4998a2c1" Oct 29 23:28:01.212345 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:01.212307 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:28:01.212424 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:01.212405 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:28:01.212475 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:01.212459 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:28:01.601359 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:01.590413 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:02.245973 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:02.245932 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:02.248690 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:02.248660 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" event={"ID":"29e6194d-41e0-4e53-8698-32ace257af0e","Type":"ContainerStarted","Data":"4afaa8cd928abf6e3b4a4d2448408e4892e9e46c3e36ce0bf11c7356d6b09b5a"} Oct 29 23:28:02.459283 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:02.458552 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:28:02.459283 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:02.458709 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Oct 29 23:28:02.459283 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:02.458759 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" podUID="29e6194d-41e0-4e53-8698-32ace257af0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Oct 29 23:28:02.836841 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:02.836767 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:03.040636 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:03.040602 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:03.449054 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:03.448611 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Oct 29 23:28:03.449054 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:03.448671 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" podUID="29e6194d-41e0-4e53-8698-32ace257af0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Oct 29 23:28:03.680862 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:03.680817 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:03.925894 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:03.925858 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:04.246339 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:04.246211 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:04.448306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:04.448061 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Oct 29 23:28:04.448306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:04.448124 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" podUID="29e6194d-41e0-4e53-8698-32ace257af0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Oct 29 23:28:05.273234 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:05.273188 5056 generic.go:356] "Generic (PLEG): container finished" podID="b46bbd7f-4b97-4b30-b500-1c31f0507898" containerID="b684f331163d46ae3a985263979830c8fd0d38f46e2c5df184da198dd480426d" exitCode=0 Oct 29 23:28:05.273925 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:05.273285 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" event={"ID":"b46bbd7f-4b97-4b30-b500-1c31f0507898","Type":"ContainerDied","Data":"b684f331163d46ae3a985263979830c8fd0d38f46e2c5df184da198dd480426d"} Oct 29 23:28:05.274516 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:05.274497 5056 scope.go:122] "RemoveContainer" containerID="b684f331163d46ae3a985263979830c8fd0d38f46e2c5df184da198dd480426d" Oct 29 23:28:05.276108 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:05.276084 5056 generic.go:356] "Generic (PLEG): container finished" podID="1a24e9f0-98ce-4367-99b2-25653f0171ee" containerID="c7c965a4faa698e70a94442ca3e2686c47cb0c57651d2d94f414611b6cf3ca6b" exitCode=0 Oct 29 23:28:05.276183 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:05.276079 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:28:05.276220 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:05.276207 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:28:05.276331 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:05.276244 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:28:05.276331 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:05.276300 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" event={"ID":"1a24e9f0-98ce-4367-99b2-25653f0171ee","Type":"ContainerDied","Data":"c7c965a4faa698e70a94442ca3e2686c47cb0c57651d2d94f414611b6cf3ca6b"} Oct 29 23:28:05.276668 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:05.276650 5056 scope.go:122] "RemoveContainer" containerID="c7c965a4faa698e70a94442ca3e2686c47cb0c57651d2d94f414611b6cf3ca6b" Oct 29 23:28:05.278451 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:05.278417 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:28:05.278571 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:05.278512 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:28:05.278607 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:05.278575 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:28:05.458051 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:05.457163 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Oct 29 23:28:05.458051 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:05.457217 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" podUID="29e6194d-41e0-4e53-8698-32ace257af0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Oct 29 23:28:06.350973 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:06.343337 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" event={"ID":"b46bbd7f-4b97-4b30-b500-1c31f0507898","Type":"ContainerStarted","Data":"750d0b2fd32ea5d6305897f9a7d15d7f239257a14124d662e98bdb381a47fbf4"} Oct 29 23:28:06.350973 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:06.348867 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" event={"ID":"1a24e9f0-98ce-4367-99b2-25653f0171ee","Type":"ContainerStarted","Data":"33da25dafe61f4d9fe53fd9107ffad5b16ffdf5ad21448598f3af69d1da459ff"} Oct 29 23:28:06.461281 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:06.453573 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Oct 29 23:28:06.461281 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:06.453636 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" podUID="29e6194d-41e0-4e53-8698-32ace257af0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Oct 29 23:28:06.779284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:06.771626 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:06.992644 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:06.991831 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:28:06.993352 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:06.992945 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Oct 29 23:28:06.993352 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:06.992999 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" podUID="b46bbd7f-4b97-4b30-b500-1c31f0507898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Oct 29 23:28:07.164465 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:07.164376 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:28:07.189291 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:07.188062 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Oct 29 23:28:07.189291 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:07.188126 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" podUID="1a24e9f0-98ce-4367-99b2-25653f0171ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Oct 29 23:28:07.489343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:07.465097 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Oct 29 23:28:07.489343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:07.465154 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" podUID="29e6194d-41e0-4e53-8698-32ace257af0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Oct 29 23:28:07.989571 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:07.989538 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Oct 29 23:28:07.989859 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:07.989817 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" podUID="b46bbd7f-4b97-4b30-b500-1c31f0507898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Oct 29 23:28:08.025325 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:08.025286 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:08.158871 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:08.158836 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Oct 29 23:28:08.159183 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:08.159133 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" podUID="1a24e9f0-98ce-4367-99b2-25653f0171ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Oct 29 23:28:08.472286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:08.471563 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:28:08.477597 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:08.477565 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:28:08.484076 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:08.483561 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:28:08.661313 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:08.660584 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:08.870353 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:08.869229 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:08.989878 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:08.989844 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Oct 29 23:28:08.990049 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:08.989907 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" podUID="b46bbd7f-4b97-4b30-b500-1c31f0507898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Oct 29 23:28:09.168750 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:09.162481 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Oct 29 23:28:09.168750 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:09.162558 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" podUID="1a24e9f0-98ce-4367-99b2-25653f0171ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Oct 29 23:28:09.357282 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:09.356861 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:09.637055 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:09.636961 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:09.991133 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:09.991095 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Oct 29 23:28:09.991571 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:09.991172 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" podUID="b46bbd7f-4b97-4b30-b500-1c31f0507898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Oct 29 23:28:10.159650 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:10.159615 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Oct 29 23:28:10.159970 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:10.159942 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" podUID="1a24e9f0-98ce-4367-99b2-25653f0171ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Oct 29 23:28:10.846407 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:10.846369 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:28:10.989927 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:10.989893 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Oct 29 23:28:10.990093 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:10.989953 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" podUID="b46bbd7f-4b97-4b30-b500-1c31f0507898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Oct 29 23:28:11.162293 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:11.161631 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Oct 29 23:28:11.162293 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:11.161691 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" podUID="1a24e9f0-98ce-4367-99b2-25653f0171ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Oct 29 23:28:11.853182 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:11.853140 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:28:12.027297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:12.026302 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:12.239792 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:12.239759 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:28:12.241464 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:12.241438 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:28:12.259147 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:12.259119 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:28:12.589599 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:12.589521 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:28:13.202068 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:13.202025 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:13.795195 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:13.795160 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:13.981885 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:13.981828 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:14.146531 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:14.146504 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:14.849136 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:14.849090 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:16.124063 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:16.124029 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:18.083636 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:18.083603 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:18.914542 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:18.914505 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:20.011473 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:20.011437 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:21.846944 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:21.846911 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:22.416411 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:28:22.416374 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:28:23.043565 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:23.043528 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:23.862123 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:23.862081 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:25.821712 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:25.818951 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:27.663916 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:27.662955 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:29.408782 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:29.408747 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:29.561723 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:29.561689 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:31.508301 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:31.508250 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:33.354830 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:33.354788 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:34.075320 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:34.075289 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:35.165911 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:35.165878 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:36.260404 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:36.258859 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:37.093958 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:37.093906 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:38.082941 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.082905 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9"] Oct 29 23:28:38.088363 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.088344 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.107119 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.107097 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-b978a0-openshift-default-dockercfg-jmbzw\"" Oct 29 23:28:38.122832 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.122807 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9"] Oct 29 23:28:38.218717 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.218679 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.218892 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.218722 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.218892 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.218869 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.218992 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.218927 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.219033 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.219014 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.219082 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.219067 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.219193 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.219172 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.219240 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.219224 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.219334 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.219307 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkrn6\" (UniqueName: \"kubernetes.io/projected/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-kube-api-access-dkrn6\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.321025 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.320988 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.321218 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.321040 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkrn6\" (UniqueName: \"kubernetes.io/projected/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-kube-api-access-dkrn6\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.321218 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.321105 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.321363 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.321334 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.321431 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.321416 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.321493 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.321458 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.321493 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.321487 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.321594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.321502 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.321594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.321543 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.321702 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.321625 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.321702 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.321688 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.321821 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.321753 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.322023 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.321998 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.322332 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.322311 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.323763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.323737 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.324168 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.324151 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.345609 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.345552 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.345957 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.345935 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkrn6\" (UniqueName: \"kubernetes.io/projected/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-kube-api-access-dkrn6\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.401986 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.401945 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:38.723143 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.723115 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9"] Oct 29 23:28:38.724206 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:28:38.724177 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35b2936b_6eff_4905_97f3_fb3ef5bdb95c.slice/crio-e4d685fb42ce925aa9e6683d8b83c4f71b42460b5dda8dc06448c1e5c0c244cd WatchSource:0}: Error finding container e4d685fb42ce925aa9e6683d8b83c4f71b42460b5dda8dc06448c1e5c0c244cd: Status 404 returned error can't find the container with id e4d685fb42ce925aa9e6683d8b83c4f71b42460b5dda8dc06448c1e5c0c244cd Oct 29 23:28:38.726527 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.726493 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:28:38.726603 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.726558 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:28:38.726603 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.726586 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:28:38.748295 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:38.748273 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" event={"ID":"35b2936b-6eff-4905-97f3-fb3ef5bdb95c","Type":"ContainerStarted","Data":"e4d685fb42ce925aa9e6683d8b83c4f71b42460b5dda8dc06448c1e5c0c244cd"} Oct 29 23:28:39.065203 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:39.065102 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:39.756072 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:39.756034 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" event={"ID":"35b2936b-6eff-4905-97f3-fb3ef5bdb95c","Type":"ContainerStarted","Data":"a6766f97617b97ab18620a9c03b3cd99a31e9c237aef125321f6a288578d26c3"} Oct 29 23:28:40.402920 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:40.402879 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:40.404510 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:40.404489 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 29 23:28:40.404579 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:40.404544 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" podUID="35b2936b-6eff-4905-97f3-fb3ef5bdb95c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 29 23:28:40.670970 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:40.670868 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:41.007863 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:41.007761 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:41.402681 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:41.402648 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 29 23:28:41.402843 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:41.402725 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" podUID="35b2936b-6eff-4905-97f3-fb3ef5bdb95c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 29 23:28:42.407703 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:42.407665 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 29 23:28:42.408362 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:42.408328 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" podUID="35b2936b-6eff-4905-97f3-fb3ef5bdb95c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 29 23:28:43.211474 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:43.207929 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:43.402422 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:43.402386 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 29 23:28:43.402615 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:43.402447 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" podUID="35b2936b-6eff-4905-97f3-fb3ef5bdb95c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 29 23:28:44.294987 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:44.294948 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:44.402974 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:44.402944 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Oct 29 23:28:44.403145 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:44.402992 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" podUID="35b2936b-6eff-4905-97f3-fb3ef5bdb95c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Oct 29 23:28:45.017978 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:45.017936 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:45.407826 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:45.407789 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:45.408242 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:45.408170 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:45.408856 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:45.408838 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:28:45.455386 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:45.455342 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" podStartSLOduration=7.455330364 podStartE2EDuration="7.455330364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:28:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:28:39.846640673 +0000 UTC m=+947.213388305" watchObservedRunningTime="2026-10-29 23:28:45.455330364 +0000 UTC m=+952.822077995" Oct 29 23:28:45.519652 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:45.519615 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:45.674294 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:45.674197 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:46.776242 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:46.776204 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:48.876830 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:48.876785 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:49.505808 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:49.505768 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:50.651270 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:50.651220 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:52.418399 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:28:52.418361 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:28:52.733868 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:52.733788 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:53.578104 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.578072 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9"] Oct 29 23:28:53.582370 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.582350 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.599052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.599029 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-53c44e-openshift-default-dockercfg-hgv77\"" Oct 29 23:28:53.616273 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.616234 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9"] Oct 29 23:28:53.638519 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.638490 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.638667 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.638534 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21f739e-967e-4f1f-81e9-679f0c549416-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.638667 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.638563 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvdkj\" (UniqueName: \"kubernetes.io/projected/d21f739e-967e-4f1f-81e9-679f0c549416-kube-api-access-pvdkj\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.638667 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.638655 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.638832 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.638818 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21f739e-967e-4f1f-81e9-679f0c549416-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.638887 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.638862 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.638941 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.638886 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21f739e-967e-4f1f-81e9-679f0c549416-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.638995 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.638970 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.639044 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.639007 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.739950 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.739912 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.740122 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.740094 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21f739e-967e-4f1f-81e9-679f0c549416-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.740166 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.740155 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.740200 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.740183 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21f739e-967e-4f1f-81e9-679f0c549416-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.740317 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.740286 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.740408 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.740326 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.740408 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.740360 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.740408 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.740388 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21f739e-967e-4f1f-81e9-679f0c549416-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.740563 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.740413 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvdkj\" (UniqueName: \"kubernetes.io/projected/d21f739e-967e-4f1f-81e9-679f0c549416-kube-api-access-pvdkj\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.740619 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.740568 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.740865 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.740837 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.740998 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.740910 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.741247 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.741227 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21f739e-967e-4f1f-81e9-679f0c549416-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.741451 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.741432 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.742397 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.742374 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.742652 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.742634 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21f739e-967e-4f1f-81e9-679f0c549416-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.768577 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.768549 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21f739e-967e-4f1f-81e9-679f0c549416-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.768676 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.768649 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvdkj\" (UniqueName: \"kubernetes.io/projected/d21f739e-967e-4f1f-81e9-679f0c549416-kube-api-access-pvdkj\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:53.896834 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:53.896802 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:54.219599 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:54.219567 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9"] Oct 29 23:28:54.221165 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:28:54.221138 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21f739e_967e_4f1f_81e9_679f0c549416.slice/crio-b873ee5c22031c577a9e0bb7561598191733c7fa6ad2ef11fd25309a22b2bbe8 WatchSource:0}: Error finding container b873ee5c22031c577a9e0bb7561598191733c7fa6ad2ef11fd25309a22b2bbe8: Status 404 returned error can't find the container with id b873ee5c22031c577a9e0bb7561598191733c7fa6ad2ef11fd25309a22b2bbe8 Oct 29 23:28:54.224019 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:54.223987 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:28:54.224086 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:54.224053 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:28:54.224086 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:54.224079 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:28:54.571334 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:54.571111 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:54.972564 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:54.968769 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" event={"ID":"d21f739e-967e-4f1f-81e9-679f0c549416","Type":"ContainerStarted","Data":"6bf3bc17c64d72b298a8641403c5b9bc0f54f68cce7391887d9205090c40876b"} Oct 29 23:28:54.972564 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:54.968893 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" event={"ID":"d21f739e-967e-4f1f-81e9-679f0c549416","Type":"ContainerStarted","Data":"b873ee5c22031c577a9e0bb7561598191733c7fa6ad2ef11fd25309a22b2bbe8"} Oct 29 23:28:55.043287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:55.040645 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" podStartSLOduration=2.040623737 podStartE2EDuration="2.040623737s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:28:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:28:55.038322067 +0000 UTC m=+962.405069703" watchObservedRunningTime="2026-10-29 23:28:55.040623737 +0000 UTC m=+962.407371370" Oct 29 23:28:55.051287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:55.050761 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:55.898102 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:55.898076 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:28:55.899774 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:55.899750 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 29 23:28:55.899894 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:55.899817 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" podUID="d21f739e-967e-4f1f-81e9-679f0c549416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 29 23:28:56.592449 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:56.592403 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:56.897699 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:56.897670 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 29 23:28:56.897863 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:56.897727 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" podUID="d21f739e-967e-4f1f-81e9-679f0c549416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 29 23:28:57.898159 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:57.898125 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 29 23:28:57.898604 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:57.898181 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" podUID="d21f739e-967e-4f1f-81e9-679f0c549416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 29 23:28:58.589508 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:58.589426 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:28:58.897370 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:58.897342 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 29 23:28:58.897539 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:58.897400 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" podUID="d21f739e-967e-4f1f-81e9-679f0c549416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 29 23:28:59.897837 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:59.897799 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 29 23:28:59.898246 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:28:59.897853 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" podUID="d21f739e-967e-4f1f-81e9-679f0c549416" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 29 23:29:00.766748 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:00.766679 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:01.756671 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:01.756353 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:29:01.757396 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:01.757360 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:29:01.758788 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:01.758761 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:29:02.453318 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:02.452852 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:04.266292 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:04.262483 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:05.323198 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:05.323165 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:06.238306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:06.238250 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:06.870450 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:06.870421 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:08.245976 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:08.245940 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:10.347866 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:10.347827 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:12.097434 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:12.097384 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:13.049894 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:13.049856 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:13.917876 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:13.917220 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:15.695317 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:15.695275 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:17.653510 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:17.653466 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:19.770432 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:19.770363 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:21.941976 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:21.941940 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:22.416637 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:29:22.416594 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:29:23.755941 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:23.755910 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z"] Oct 29 23:29:23.760066 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:23.760036 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:23.786543 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:23.786516 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z"] Oct 29 23:29:23.889652 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:23.889614 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:23.889652 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:23.889657 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:23.889894 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:23.889760 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:23.889894 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:23.889808 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:23.889968 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:23.889897 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2lb2\" (UniqueName: \"kubernetes.io/projected/e1c93822-3b0a-4f84-a23d-26e04ffc4805-kube-api-access-j2lb2\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:23.889968 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:23.889928 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:23.890042 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:23.890005 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:23.890082 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:23.890043 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:23.890132 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:23.890093 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.003280 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.003120 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.003280 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.003189 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.003280 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.003282 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.003574 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.003317 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.008315 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.003377 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2lb2\" (UniqueName: \"kubernetes.io/projected/e1c93822-3b0a-4f84-a23d-26e04ffc4805-kube-api-access-j2lb2\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.008463 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.008419 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.008527 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.008491 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.008588 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.008541 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.008644 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.008588 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.008982 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.008957 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.009128 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.009003 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.009555 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.009531 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.009733 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.009709 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.010040 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.010020 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.015936 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.015910 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.017131 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.017107 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.035918 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.035891 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.036768 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.036744 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2lb2\" (UniqueName: \"kubernetes.io/projected/e1c93822-3b0a-4f84-a23d-26e04ffc4805-kube-api-access-j2lb2\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.042752 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.042721 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:24.074852 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.074823 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:24.438589 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.438560 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z"] Oct 29 23:29:24.439424 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:29:24.439382 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1c93822_3b0a_4f84_a23d_26e04ffc4805.slice/crio-ed90f5bfb935565a2d9da5749446c07b07f2960f5c4d4c549cfb67841c31280c WatchSource:0}: Error finding container ed90f5bfb935565a2d9da5749446c07b07f2960f5c4d4c549cfb67841c31280c: Status 404 returned error can't find the container with id ed90f5bfb935565a2d9da5749446c07b07f2960f5c4d4c549cfb67841c31280c Oct 29 23:29:24.441933 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.441899 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:29:24.442019 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.441962 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:29:24.442019 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:24.441991 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:29:25.182950 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:25.182909 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" event={"ID":"e1c93822-3b0a-4f84-a23d-26e04ffc4805","Type":"ContainerStarted","Data":"6aa79afc6dc04281c395377d0530d243ca28b9f7a0f4be6c059e7964a7c50cf9"} Oct 29 23:29:25.183418 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:25.182988 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" event={"ID":"e1c93822-3b0a-4f84-a23d-26e04ffc4805","Type":"ContainerStarted","Data":"ed90f5bfb935565a2d9da5749446c07b07f2960f5c4d4c549cfb67841c31280c"} Oct 29 23:29:25.249086 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:25.249035 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" podStartSLOduration=2.249022341 podStartE2EDuration="2.249022341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:29:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:29:25.248519872 +0000 UTC m=+992.615267503" watchObservedRunningTime="2026-10-29 23:29:25.249022341 +0000 UTC m=+992.615769971" Oct 29 23:29:26.075423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:26.075387 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:26.077251 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:26.077215 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 29 23:29:26.077400 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:26.077278 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" podUID="e1c93822-3b0a-4f84-a23d-26e04ffc4805" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 29 23:29:26.256353 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:26.256306 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:27.075516 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:27.075478 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 29 23:29:27.075693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:27.075540 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" podUID="e1c93822-3b0a-4f84-a23d-26e04ffc4805" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 29 23:29:28.078743 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:28.078710 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 29 23:29:28.079372 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:28.079324 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" podUID="e1c93822-3b0a-4f84-a23d-26e04ffc4805" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 29 23:29:28.081300 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:28.080703 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:29.075949 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:29.075919 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 29 23:29:29.076186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:29.076161 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" podUID="e1c93822-3b0a-4f84-a23d-26e04ffc4805" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 29 23:29:30.076118 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:30.076076 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body= Oct 29 23:29:30.076625 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:30.076598 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" podUID="e1c93822-3b0a-4f84-a23d-26e04ffc4805" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" Oct 29 23:29:30.277765 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:30.277729 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:31.119870 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:31.119492 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:31.120779 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:31.120757 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:31.122096 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:31.122077 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:29:32.176320 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:32.175177 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:34.247395 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:34.247360 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:36.241761 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:36.241680 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:37.954024 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:37.953836 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:38.841014 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:38.840976 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg"] Oct 29 23:29:38.847325 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:38.847298 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:38.871176 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:38.871146 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg"] Oct 29 23:29:38.923644 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:38.923615 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjjdc\" (UniqueName: \"kubernetes.io/projected/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-kube-api-access-tjjdc\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:38.923852 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:38.923660 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:38.923852 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:38.923727 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:38.923852 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:38.923813 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:38.924008 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:38.923858 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:38.924008 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:38.923903 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:38.924008 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:38.923998 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:38.924120 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:38.924056 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:38.924157 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:38.924128 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:38.961283 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:38.961226 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-aa7e96-openshift-default-dockercfg-8q4lr\"" Oct 29 23:29:39.025616 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.025577 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.025816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.025625 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.025816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.025725 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjjdc\" (UniqueName: \"kubernetes.io/projected/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-kube-api-access-tjjdc\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.025816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.025769 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.025816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.025801 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.026026 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.025831 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.026026 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.025861 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.026026 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.025902 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.026026 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.025982 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.026327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.026298 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.026473 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.026306 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.026473 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.026452 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.026591 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.026499 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.026671 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.026648 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.028380 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.028357 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.028632 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.028615 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.050603 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.050577 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.050752 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.050740 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjjdc\" (UniqueName: \"kubernetes.io/projected/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-kube-api-access-tjjdc\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.162150 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.162119 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:39.407612 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.407578 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt"] Oct 29 23:29:39.420353 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.414388 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.456415 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.456383 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt"] Oct 29 23:29:39.500865 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.500836 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg"] Oct 29 23:29:39.501374 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:29:39.501351 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb5ef72_5ab0_4f03_9e6e_a0cc964377f6.slice/crio-61da217ab1f461ce04e94106bb13b806018922eb447fcfca542ebfc7bfddf8d9 WatchSource:0}: Error finding container 61da217ab1f461ce04e94106bb13b806018922eb447fcfca542ebfc7bfddf8d9: Status 404 returned error can't find the container with id 61da217ab1f461ce04e94106bb13b806018922eb447fcfca542ebfc7bfddf8d9 Oct 29 23:29:39.503714 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.503683 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:29:39.503779 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.503744 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:29:39.503822 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.503775 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:29:39.543951 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.543926 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h2p4\" (UniqueName: \"kubernetes.io/projected/7e1dad7e-beb0-49c7-be70-d99766042790-kube-api-access-7h2p4\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.544062 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.543969 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.544062 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.543999 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e1dad7e-beb0-49c7-be70-d99766042790-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.544162 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.544137 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.544206 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.544192 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.544279 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.544251 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.544342 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.544328 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e1dad7e-beb0-49c7-be70-d99766042790-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.544412 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.544398 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.544454 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.544427 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e1dad7e-beb0-49c7-be70-d99766042790-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.645777 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.645739 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.645981 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.645798 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.645981 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.645852 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.645981 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.645907 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e1dad7e-beb0-49c7-be70-d99766042790-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.645981 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.645969 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.646185 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.645994 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e1dad7e-beb0-49c7-be70-d99766042790-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.646185 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.646069 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h2p4\" (UniqueName: \"kubernetes.io/projected/7e1dad7e-beb0-49c7-be70-d99766042790-kube-api-access-7h2p4\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.646185 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.646098 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.646185 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.646127 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e1dad7e-beb0-49c7-be70-d99766042790-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.646185 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.646177 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.646460 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.646437 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.646696 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.646678 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.647321 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.647295 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.647531 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.647503 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e1dad7e-beb0-49c7-be70-d99766042790-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.649176 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.649152 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e1dad7e-beb0-49c7-be70-d99766042790-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.649405 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.649386 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.671559 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.671500 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e1dad7e-beb0-49c7-be70-d99766042790-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.671559 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.671533 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h2p4\" (UniqueName: \"kubernetes.io/projected/7e1dad7e-beb0-49c7-be70-d99766042790-kube-api-access-7h2p4\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.770965 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.770922 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:39.945343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:39.945192 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:40.105176 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:29:40.105148 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e1dad7e_beb0_49c7_be70_d99766042790.slice/crio-803daf11eb389d3311c37ca18c91bd9eff0ed7bdce855556c113f5dad1fb6ceb WatchSource:0}: Error finding container 803daf11eb389d3311c37ca18c91bd9eff0ed7bdce855556c113f5dad1fb6ceb: Status 404 returned error can't find the container with id 803daf11eb389d3311c37ca18c91bd9eff0ed7bdce855556c113f5dad1fb6ceb Oct 29 23:29:40.105176 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:40.105152 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt"] Oct 29 23:29:40.107883 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:40.107843 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:29:40.107981 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:40.107931 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:29:40.107981 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:40.107974 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:29:40.356331 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:40.356109 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" event={"ID":"7e1dad7e-beb0-49c7-be70-d99766042790","Type":"ContainerStarted","Data":"de2fd2b39bb6ebdc532019edc3e4712feb3c6c93a04f5d79c5748808349dce87"} Oct 29 23:29:40.356331 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:40.356181 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" event={"ID":"7e1dad7e-beb0-49c7-be70-d99766042790","Type":"ContainerStarted","Data":"803daf11eb389d3311c37ca18c91bd9eff0ed7bdce855556c113f5dad1fb6ceb"} Oct 29 23:29:40.362303 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:40.359727 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" event={"ID":"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6","Type":"ContainerStarted","Data":"c1480c0611c17833c86ea4a30c1c2c4ab2e828ba32ad55bb2e66e86492bf09ee"} Oct 29 23:29:40.362303 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:40.359783 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" event={"ID":"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6","Type":"ContainerStarted","Data":"61da217ab1f461ce04e94106bb13b806018922eb447fcfca542ebfc7bfddf8d9"} Oct 29 23:29:40.408920 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:40.408868 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" podStartSLOduration=1.408853474 podStartE2EDuration="1.408853474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:29:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:29:40.406765774 +0000 UTC m=+1007.773513406" watchObservedRunningTime="2026-10-29 23:29:40.408853474 +0000 UTC m=+1007.775601163" Oct 29 23:29:40.771337 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:40.771307 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:41.174287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:41.165371 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:41.174287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:41.165517 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 29 23:29:41.174287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:41.165563 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" podUID="6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 29 23:29:41.778439 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:41.778403 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 29 23:29:41.778697 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:41.778670 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" podUID="7e1dad7e-beb0-49c7-be70-d99766042790" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 29 23:29:41.917287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:41.915170 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:42.164287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:42.163615 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 29 23:29:42.164287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:42.163676 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" podUID="6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 29 23:29:42.773172 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:42.773136 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 29 23:29:42.773640 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:42.773366 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" podUID="7e1dad7e-beb0-49c7-be70-d99766042790" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 29 23:29:43.172934 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:43.172500 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 29 23:29:43.172934 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:43.172556 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" podUID="6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 29 23:29:43.776427 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:43.776383 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 29 23:29:43.777029 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:43.776989 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" podUID="7e1dad7e-beb0-49c7-be70-d99766042790" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 29 23:29:44.174189 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:44.173714 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 29 23:29:44.174189 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:44.173773 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" podUID="6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 29 23:29:44.587340 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:44.587042 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:44.999285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:44.999157 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 29 23:29:44.999285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:44.999218 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" podUID="7e1dad7e-beb0-49c7-be70-d99766042790" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 29 23:29:45.057286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:45.038183 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:45.168693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:45.168654 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 29 23:29:45.168963 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:45.168934 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" podUID="6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 29 23:29:45.776286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:45.773623 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 29 23:29:45.776286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:45.773701 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" podUID="7e1dad7e-beb0-49c7-be70-d99766042790" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 29 23:29:46.165072 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:46.164639 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body= Oct 29 23:29:46.165072 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:46.164704 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" podUID="6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" Oct 29 23:29:46.771547 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:46.771509 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 29 23:29:46.771726 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:46.771581 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" podUID="7e1dad7e-beb0-49c7-be70-d99766042790" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 29 23:29:47.632256 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:47.632164 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:47.771276 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:47.771215 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 29 23:29:47.771448 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:47.771296 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" podUID="7e1dad7e-beb0-49c7-be70-d99766042790" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 29 23:29:48.209464 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:48.209423 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:29:48.209651 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:48.209495 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" podUID="6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:29:48.442020 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:48.441987 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:48.442487 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:48.442464 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:48.443193 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:48.443169 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:29:48.507496 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:48.507374 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" podStartSLOduration=10.507359158 podStartE2EDuration="10.507359158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:29:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:29:40.463284651 +0000 UTC m=+1007.830032281" watchObservedRunningTime="2026-10-29 23:29:48.507359158 +0000 UTC m=+1015.874106790" Oct 29 23:29:48.775899 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:48.775829 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:48.776324 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:48.776155 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:48.776985 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:48.776970 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:29:49.744879 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:49.744836 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:51.540293 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:51.531808 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:52.417323 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:29:52.417288 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:29:53.890044 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:53.889880 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:55.920816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:55.920777 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:57.791313 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:57.791214 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:29:59.737508 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:29:59.737463 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:02.178494 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:02.178439 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:30:02.179160 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:02.179125 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" podUID="e1c93822-3b0a-4f84-a23d-26e04ffc4805" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:30:03.105998 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:03.105958 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:04.044285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:04.040773 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:04.419727 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:04.419369 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-9k9s2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:30:04.419727 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:04.419447 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" podUID="a27239f9-531e-4f44-be00-22c11b243a3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:30:05.868764 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:05.868688 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:07.973912 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:07.973872 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:08.569167 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.569133 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47"] Oct 29 23:30:08.574757 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.574733 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.581720 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.581692 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.581878 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.581723 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6qp4\" (UniqueName: \"kubernetes.io/projected/f0bb96ed-9f9c-46de-b003-fa0e7c199259-kube-api-access-x6qp4\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.581878 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.581748 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.581878 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.581781 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.582048 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.581989 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.582048 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.582024 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.582190 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.582162 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.582302 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.582251 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.582402 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.582380 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.613591 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.613553 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47"] Oct 29 23:30:08.683733 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.683692 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.683733 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.683726 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.684005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.683753 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.684005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.683786 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.684005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.683835 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.684005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.683911 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.684005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.683937 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6qp4\" (UniqueName: \"kubernetes.io/projected/f0bb96ed-9f9c-46de-b003-fa0e7c199259-kube-api-access-x6qp4\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.684005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.683969 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.684005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.684004 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.684399 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.684219 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.684399 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.684241 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.684531 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.684508 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.684592 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.684547 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.684647 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.684635 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.686429 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.686404 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.686630 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.686614 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.715338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.715309 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6qp4\" (UniqueName: \"kubernetes.io/projected/f0bb96ed-9f9c-46de-b003-fa0e7c199259-kube-api-access-x6qp4\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.715338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.715325 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:08.890010 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:08.889975 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:09.235543 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:09.235505 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47"] Oct 29 23:30:09.235971 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:30:09.235841 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0bb96ed_9f9c_46de_b003_fa0e7c199259.slice/crio-e77ff0fe62d1422c484d175ef1e61c986f164367d03497926015a0bcc4564ec4 WatchSource:0}: Error finding container e77ff0fe62d1422c484d175ef1e61c986f164367d03497926015a0bcc4564ec4: Status 404 returned error can't find the container with id e77ff0fe62d1422c484d175ef1e61c986f164367d03497926015a0bcc4564ec4 Oct 29 23:30:09.238636 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:09.238602 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:30:09.238723 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:09.238668 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:30:09.238723 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:09.238696 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:30:09.803924 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:09.803888 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" event={"ID":"f0bb96ed-9f9c-46de-b003-fa0e7c199259","Type":"ContainerStarted","Data":"7eca9ca68238258fa851a056cbdd5758bd71cdd4e79625a21d475afe038d8e1d"} Oct 29 23:30:09.804092 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:09.803947 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" event={"ID":"f0bb96ed-9f9c-46de-b003-fa0e7c199259","Type":"ContainerStarted","Data":"e77ff0fe62d1422c484d175ef1e61c986f164367d03497926015a0bcc4564ec4"} Oct 29 23:30:09.884762 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:09.884710 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" podStartSLOduration=1.8846957519999998 podStartE2EDuration="1.884695752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:09.870988925 +0000 UTC m=+1037.237736553" watchObservedRunningTime="2026-10-29 23:30:09.884695752 +0000 UTC m=+1037.251443382" Oct 29 23:30:09.890754 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:09.890717 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:10.011592 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:10.010565 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:10.892233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:10.892198 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 29 23:30:10.892738 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:10.892255 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" podUID="f0bb96ed-9f9c-46de-b003-fa0e7c199259" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 29 23:30:11.891068 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:11.891028 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 29 23:30:11.891243 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:11.891094 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" podUID="f0bb96ed-9f9c-46de-b003-fa0e7c199259" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 29 23:30:12.075222 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:12.073253 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:12.891027 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:12.890994 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 29 23:30:12.891209 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:12.891052 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" podUID="f0bb96ed-9f9c-46de-b003-fa0e7c199259" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 29 23:30:13.890904 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:13.890872 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 29 23:30:13.891381 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:13.890933 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" podUID="f0bb96ed-9f9c-46de-b003-fa0e7c199259" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 29 23:30:13.945307 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:13.937121 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:14.890772 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:14.890740 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 29 23:30:14.890940 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:14.890789 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" podUID="f0bb96ed-9f9c-46de-b003-fa0e7c199259" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 29 23:30:15.851001 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:15.850963 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:15.902040 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:15.902009 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:15.902909 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:15.902827 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:15.903540 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:15.903515 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:30:18.672295 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:18.670979 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:20.536805 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:20.536767 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:22.309794 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.309753 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:22.416979 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:30:22.416944 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:30:22.674877 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.674842 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t"] Oct 29 23:30:22.679320 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.679296 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.717252 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.717222 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t"] Oct 29 23:30:22.774853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.774821 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d81526c4-5f8d-4833-beac-6cc8507caba9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.774998 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.774950 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9krc\" (UniqueName: \"kubernetes.io/projected/d81526c4-5f8d-4833-beac-6cc8507caba9-kube-api-access-l9krc\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.774998 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.774986 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d81526c4-5f8d-4833-beac-6cc8507caba9-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.775084 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.775019 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d81526c4-5f8d-4833-beac-6cc8507caba9-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.775084 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.775043 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d81526c4-5f8d-4833-beac-6cc8507caba9-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.775151 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.775097 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d81526c4-5f8d-4833-beac-6cc8507caba9-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.775151 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.775136 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d81526c4-5f8d-4833-beac-6cc8507caba9-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.775212 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.775162 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d81526c4-5f8d-4833-beac-6cc8507caba9-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.775247 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.775213 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d81526c4-5f8d-4833-beac-6cc8507caba9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.876864 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.876814 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d81526c4-5f8d-4833-beac-6cc8507caba9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.877052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.877010 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9krc\" (UniqueName: \"kubernetes.io/projected/d81526c4-5f8d-4833-beac-6cc8507caba9-kube-api-access-l9krc\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.877115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.877050 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d81526c4-5f8d-4833-beac-6cc8507caba9-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.877169 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.877113 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d81526c4-5f8d-4833-beac-6cc8507caba9-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.877169 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.877138 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d81526c4-5f8d-4833-beac-6cc8507caba9-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.877289 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.877201 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d81526c4-5f8d-4833-beac-6cc8507caba9-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.877346 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.877306 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d81526c4-5f8d-4833-beac-6cc8507caba9-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.877399 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.877346 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d81526c4-5f8d-4833-beac-6cc8507caba9-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.877455 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.877408 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d81526c4-5f8d-4833-beac-6cc8507caba9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.877633 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.877612 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d81526c4-5f8d-4833-beac-6cc8507caba9-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.878185 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.878163 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d81526c4-5f8d-4833-beac-6cc8507caba9-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.878560 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.878435 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d81526c4-5f8d-4833-beac-6cc8507caba9-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.878560 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.878547 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d81526c4-5f8d-4833-beac-6cc8507caba9-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.878734 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.878689 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d81526c4-5f8d-4833-beac-6cc8507caba9-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.881123 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.881097 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d81526c4-5f8d-4833-beac-6cc8507caba9-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.881705 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.881684 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d81526c4-5f8d-4833-beac-6cc8507caba9-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.902814 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.902787 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d81526c4-5f8d-4833-beac-6cc8507caba9-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.902988 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.902965 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9krc\" (UniqueName: \"kubernetes.io/projected/d81526c4-5f8d-4833-beac-6cc8507caba9-kube-api-access-l9krc\") pod \"maas-default-gateway-openshift-default-746ff8f49f-blm5t\" (UID: \"d81526c4-5f8d-4833-beac-6cc8507caba9\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:22.996051 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:22.995973 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:23.324061 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:23.324025 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t"] Oct 29 23:30:23.324427 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:30:23.324027 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd81526c4_5f8d_4833_beac_6cc8507caba9.slice/crio-37c02ecc7cc7249aa07d40d991d678fbf90367be56fcba768b2873fd6c28d0d9 WatchSource:0}: Error finding container 37c02ecc7cc7249aa07d40d991d678fbf90367be56fcba768b2873fd6c28d0d9: Status 404 returned error can't find the container with id 37c02ecc7cc7249aa07d40d991d678fbf90367be56fcba768b2873fd6c28d0d9 Oct 29 23:30:23.326596 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:23.326564 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:30:23.326693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:23.326632 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:30:23.326693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:23.326658 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:30:23.977709 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:23.977674 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" event={"ID":"d81526c4-5f8d-4833-beac-6cc8507caba9","Type":"ContainerStarted","Data":"797f81ee41a98f3e22c673b8780e85b42d3d6d9f8e38bc3d760ae9f39c73a1c3"} Oct 29 23:30:23.977908 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:23.977734 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" event={"ID":"d81526c4-5f8d-4833-beac-6cc8507caba9","Type":"ContainerStarted","Data":"37c02ecc7cc7249aa07d40d991d678fbf90367be56fcba768b2873fd6c28d0d9"} Oct 29 23:30:23.996873 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:23.996849 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:24.391337 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.389221 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:24.475305 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.468922 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" podStartSLOduration=2.468900231 podStartE2EDuration="2.468900231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:24.034559472 +0000 UTC m=+1051.401307104" watchObservedRunningTime="2026-10-29 23:30:24.468900231 +0000 UTC m=+1051.835647865" Oct 29 23:30:24.482655 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.480611 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6"] Oct 29 23:30:24.488724 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.487587 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.504419 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.504397 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6"] Oct 29 23:30:24.508374 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.508343 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-53c44e\"/\"openshift-service-ca.crt\"" Oct 29 23:30:24.508374 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.508346 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-53c44e\"/\"default-dockercfg-z6kgj\"" Oct 29 23:30:24.508545 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.508408 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-53c44e\"/\"distinct-w5-53c44e-kserve-self-signed-certs\"" Oct 29 23:30:24.508706 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.508686 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-53c44e\"/\"kube-root-ca.crt\"" Oct 29 23:30:24.571640 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.571609 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w"] Oct 29 23:30:24.575903 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.575884 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.590345 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.590321 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:24.597477 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.597449 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnjbv\" (UniqueName: \"kubernetes.io/projected/9d0d46e2-581f-4dc5-8888-ee516232efde-kube-api-access-gnjbv\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.597609 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.597493 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-dshm\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.597678 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.597648 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-tmp-dir\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.597741 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.597688 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d0d46e2-581f-4dc5-8888-ee516232efde-tls-certs\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.597851 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.597833 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-home\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.597931 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.597906 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-model-cache\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.604527 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.604479 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w"] Oct 29 23:30:24.792338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.768422 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.792338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.780155 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.792338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.780200 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-home\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.792338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.780283 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.792338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.780335 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-model-cache\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.792338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.780414 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.792338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.780459 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.792338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.780492 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnjbv\" (UniqueName: \"kubernetes.io/projected/9d0d46e2-581f-4dc5-8888-ee516232efde-kube-api-access-gnjbv\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.792338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.780518 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-dshm\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.792338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.780550 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9b9lc\" (UniqueName: \"kubernetes.io/projected/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-kube-api-access-9b9lc\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.792338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.782177 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-home\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.792338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.782484 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-model-cache\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.792338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.790677 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-tmp-dir\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.837286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.834372 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-tmp-dir\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.837286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.834444 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.837286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.834473 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.837286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.834519 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d0d46e2-581f-4dc5-8888-ee516232efde-tls-certs\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.837286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.834544 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.862281 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.855923 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d0d46e2-581f-4dc5-8888-ee516232efde-tls-certs\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.862281 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.859596 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-dshm\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.868971 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.868901 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnjbv\" (UniqueName: \"kubernetes.io/projected/9d0d46e2-581f-4dc5-8888-ee516232efde-kube-api-access-gnjbv\") pod \"distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:24.946285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.941015 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.946285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.941074 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.946285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.941116 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9b9lc\" (UniqueName: \"kubernetes.io/projected/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-kube-api-access-9b9lc\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.946285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.941191 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.946285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.941219 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.946285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.941254 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.946285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.941314 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.946285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.941411 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.946285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.941474 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.946285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.941960 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.946285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.942973 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.946285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.943832 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.946285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.944621 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.950651 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.950539 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.953560 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.953541 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.953795 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.953770 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.967747 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.967699 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9b9lc\" (UniqueName: \"kubernetes.io/projected/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-kube-api-access-9b9lc\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.967888 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.967866 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:24.997527 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.997497 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-blm5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 29 23:30:24.997654 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:24.997552 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" podUID="d81526c4-5f8d-4833-beac-6cc8507caba9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 29 23:30:25.112247 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:25.112165 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:25.193322 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:25.192529 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:25.472040 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:25.471950 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6"] Oct 29 23:30:25.575896 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:25.575868 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w"] Oct 29 23:30:25.576150 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:30:25.576123 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6e9182f_276e_4ca6_bf97_16a1cf21ba00.slice/crio-1141907abf5962e1a6507409321c61ce78c26dbd7c0b55b977b08304b6311d4c WatchSource:0}: Error finding container 1141907abf5962e1a6507409321c61ce78c26dbd7c0b55b977b08304b6311d4c: Status 404 returned error can't find the container with id 1141907abf5962e1a6507409321c61ce78c26dbd7c0b55b977b08304b6311d4c Oct 29 23:30:25.578804 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:25.578762 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:30:25.578881 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:25.578860 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:30:25.578945 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:25.578914 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:30:25.995578 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:25.995533 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" event={"ID":"b6e9182f-276e-4ca6-bf97-16a1cf21ba00","Type":"ContainerStarted","Data":"1fa9b3b5e91f9925de87dc7bf346c65a6825f6462ae86ef37ddf1cb9687de5b9"} Oct 29 23:30:25.995788 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:25.995604 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" event={"ID":"b6e9182f-276e-4ca6-bf97-16a1cf21ba00","Type":"ContainerStarted","Data":"1141907abf5962e1a6507409321c61ce78c26dbd7c0b55b977b08304b6311d4c"} Oct 29 23:30:25.998771 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:25.998743 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-blm5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 29 23:30:25.998978 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:25.998954 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" podUID="d81526c4-5f8d-4833-beac-6cc8507caba9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 29 23:30:25.999891 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:25.999799 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" event={"ID":"9d0d46e2-581f-4dc5-8888-ee516232efde","Type":"ContainerStarted","Data":"586eb0433f5c5122cb60a12acdebf8bbba1fa0bf4d8a61bf4a415ea3a994af2f"} Oct 29 23:30:25.999891 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:25.999853 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" event={"ID":"9d0d46e2-581f-4dc5-8888-ee516232efde","Type":"ContainerStarted","Data":"2ded450a0258c8b6dbede3dc5d883483bb3ca1d0c5aee7f8da7bf9285bdf2f39"} Oct 29 23:30:26.105121 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:26.105072 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" podStartSLOduration=2.105058202 podStartE2EDuration="2.105058202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:26.103398882 +0000 UTC m=+1053.470146519" watchObservedRunningTime="2026-10-29 23:30:26.105058202 +0000 UTC m=+1053.471805836" Oct 29 23:30:26.105709 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:26.105683 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" podStartSLOduration=2.105676622 podStartE2EDuration="2.105676622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:26.058646316 +0000 UTC m=+1053.425393947" watchObservedRunningTime="2026-10-29 23:30:26.105676622 +0000 UTC m=+1053.472424252" Oct 29 23:30:26.112807 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:26.112782 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:26.192855 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:26.192825 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:26.194379 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:26.194355 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 29 23:30:26.194577 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:26.194549 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" podUID="b6e9182f-276e-4ca6-bf97-16a1cf21ba00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 29 23:30:26.562813 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:26.562765 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:26.997137 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:26.997090 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-blm5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 29 23:30:26.997342 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:26.997157 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" podUID="d81526c4-5f8d-4833-beac-6cc8507caba9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 29 23:30:27.194023 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:27.193981 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 29 23:30:27.194224 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:27.194038 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" podUID="b6e9182f-276e-4ca6-bf97-16a1cf21ba00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 29 23:30:27.996578 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:27.996547 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-blm5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 29 23:30:27.997026 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:27.997002 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" podUID="d81526c4-5f8d-4833-beac-6cc8507caba9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 29 23:30:28.192992 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:28.192955 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 29 23:30:28.193161 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:28.193008 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" podUID="b6e9182f-276e-4ca6-bf97-16a1cf21ba00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 29 23:30:28.597529 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:28.596495 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:28.812693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:28.812611 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:28.998406 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:28.998373 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-blm5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 29 23:30:28.998983 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:28.998953 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" podUID="d81526c4-5f8d-4833-beac-6cc8507caba9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 29 23:30:29.193663 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:29.193627 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 29 23:30:29.193842 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:29.193676 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" podUID="b6e9182f-276e-4ca6-bf97-16a1cf21ba00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 29 23:30:29.998289 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:29.998234 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-blm5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 29 23:30:29.999059 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:29.998536 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" podUID="d81526c4-5f8d-4833-beac-6cc8507caba9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 29 23:30:30.193706 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:30.193666 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 29 23:30:30.193874 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:30.193722 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" podUID="b6e9182f-276e-4ca6-bf97-16a1cf21ba00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 29 23:30:30.998529 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:30.998494 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-blm5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 29 23:30:30.999528 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:30.999483 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" podUID="d81526c4-5f8d-4833-beac-6cc8507caba9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 29 23:30:31.193748 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:31.193711 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 29 23:30:31.193947 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:31.193769 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" podUID="b6e9182f-276e-4ca6-bf97-16a1cf21ba00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 29 23:30:31.256191 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:31.256088 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:32.193677 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:32.193633 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 29 23:30:32.194082 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:32.193704 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" podUID="b6e9182f-276e-4ca6-bf97-16a1cf21ba00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 29 23:30:32.894301 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:32.891713 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:32.894301 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:32.892425 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:32.894301 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:32.893300 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" Oct 29 23:30:33.188490 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:33.188388 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:33.886071 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:33.886036 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:33.887103 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:33.886579 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:33.888044 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:33.888018 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:30:35.008420 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:35.008382 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:35.112531 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:35.112492 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:35.131657 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:35.131629 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:35.160545 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:35.160518 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:30:36.944304 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:36.944198 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:38.791297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:38.784928 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:40.529283 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:40.523908 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:42.537624 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:42.537556 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:44.776026 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:44.775975 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:44.972314 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:44.970361 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:46.952083 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:46.952045 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:48.085833 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.085797 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6"] Oct 29 23:30:48.089703 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.089680 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.109623 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.109603 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-92f6b3\"/\"kube-root-ca.crt\"" Oct 29 23:30:48.124693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.123091 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-92f6b3\"/\"distinct-2-w3-92f6b3-kserve-self-signed-certs\"" Oct 29 23:30:48.124693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.123158 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-92f6b3\"/\"default-dockercfg-dtsbz\"" Oct 29 23:30:48.127717 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.127691 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6"] Oct 29 23:30:48.127839 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.127804 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-92f6b3\"/\"openshift-service-ca.crt\"" Oct 29 23:30:48.204454 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.204382 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:48.210746 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.210716 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-dshm\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.210853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.210760 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-home\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.210853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.210780 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-tls-certs\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.210939 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.210926 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhh8k\" (UniqueName: \"kubernetes.io/projected/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-kube-api-access-dhh8k\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.210980 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.210973 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-tmp-dir\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.211032 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.211021 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-model-cache\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.312150 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.312106 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-dshm\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.312918 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.312892 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-home\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.313442 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.313424 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-tls-certs\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.325281 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.313668 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhh8k\" (UniqueName: \"kubernetes.io/projected/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-kube-api-access-dhh8k\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.325619 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.325588 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-tmp-dir\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.326207 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.326184 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-model-cache\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.326555 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.326068 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-tmp-dir\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.326732 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.313384 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-home\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.326951 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.326928 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-model-cache\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.345717 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.342242 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-dshm\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.348068 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.348036 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhh8k\" (UniqueName: \"kubernetes.io/projected/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-kube-api-access-dhh8k\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.355990 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.355957 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-tls-certs\") pod \"distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.404912 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.404870 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:48.752030 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:48.751997 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6"] Oct 29 23:30:48.752848 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:30:48.752821 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89f122d0_9b8d_40fc_90d5_cb04ea07f31c.slice/crio-c04868e2d6093b019715b422888d3d025599bda56ff1dd4eff513b6234de2b53 WatchSource:0}: Error finding container c04868e2d6093b019715b422888d3d025599bda56ff1dd4eff513b6234de2b53: Status 404 returned error can't find the container with id c04868e2d6093b019715b422888d3d025599bda56ff1dd4eff513b6234de2b53 Oct 29 23:30:49.030707 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:49.030596 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:49.265103 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:49.265064 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" event={"ID":"89f122d0-9b8d-40fc-90d5-cb04ea07f31c","Type":"ContainerStarted","Data":"78e79340962bcb1ca2c4d7a26f56409579d8790da15f2a0af51ef43a5dfd61ca"} Oct 29 23:30:49.265103 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:49.265110 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" event={"ID":"89f122d0-9b8d-40fc-90d5-cb04ea07f31c","Type":"ContainerStarted","Data":"c04868e2d6093b019715b422888d3d025599bda56ff1dd4eff513b6234de2b53"} Oct 29 23:30:49.316691 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:49.316574 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podStartSLOduration=1.316561049 podStartE2EDuration="1.316561049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:49.314562611 +0000 UTC m=+1076.681310239" watchObservedRunningTime="2026-10-29 23:30:49.316561049 +0000 UTC m=+1076.683308680" Oct 29 23:30:49.405919 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:49.405878 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:50.918984 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:50.918943 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:52.417240 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:30:52.417199 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:30:52.633305 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:52.631079 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:53.592424 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.592389 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw"] Oct 29 23:30:53.596603 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.596586 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.627939 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.627910 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw"] Oct 29 23:30:53.695376 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.695343 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.695532 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.695385 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.695532 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.695409 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.695532 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.695473 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.695638 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.695522 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.695638 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.695585 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.695704 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.695651 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.695786 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.695772 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.695838 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.695819 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hz7p\" (UniqueName: \"kubernetes.io/projected/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-kube-api-access-6hz7p\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.796412 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.796380 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.796581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.796430 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.796581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.796555 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.796700 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.796604 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hz7p\" (UniqueName: \"kubernetes.io/projected/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-kube-api-access-6hz7p\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.796700 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.796664 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.796816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.796703 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.796816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.796734 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.796816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.796758 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.796816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.796788 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.797033 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.796936 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.797091 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.797058 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.797191 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.797156 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.797418 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.797399 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.797647 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.797630 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.798877 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.798858 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.799398 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.799379 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.823719 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.823689 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hz7p\" (UniqueName: \"kubernetes.io/projected/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-kube-api-access-6hz7p\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.824590 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.824568 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:53.912061 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:53.912035 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:54.048372 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.048336 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm"] Oct 29 23:30:54.056768 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.056732 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.060196 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.060166 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm"] Oct 29 23:30:54.101385 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.101348 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.101543 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.101416 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhdch\" (UniqueName: \"kubernetes.io/projected/052a18a8-5808-481b-a2db-6a745812f898-kube-api-access-mhdch\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.101591 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.101539 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.101638 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.101608 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.101678 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.101664 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/052a18a8-5808-481b-a2db-6a745812f898-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.101769 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.101753 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/052a18a8-5808-481b-a2db-6a745812f898-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.101813 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.101785 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.101953 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.101934 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.102006 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.101992 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/052a18a8-5808-481b-a2db-6a745812f898-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.203652 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.203569 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.203652 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.203610 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/052a18a8-5808-481b-a2db-6a745812f898-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.203652 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.203631 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/052a18a8-5808-481b-a2db-6a745812f898-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.203957 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.203658 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.203957 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.203769 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.203957 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.203827 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/052a18a8-5808-481b-a2db-6a745812f898-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.203957 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.203935 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.204183 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.203977 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhdch\" (UniqueName: \"kubernetes.io/projected/052a18a8-5808-481b-a2db-6a745812f898-kube-api-access-mhdch\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.204183 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.204028 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.204183 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.204043 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-credential-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.204183 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.204107 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-workload-certs\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.204428 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.204371 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-istio-data\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.204490 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.204454 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-workload-socket\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.204695 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.204664 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/052a18a8-5808-481b-a2db-6a745812f898-istiod-ca-cert\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.206188 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.206163 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-istio-envoy\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.206313 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.206234 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/052a18a8-5808-481b-a2db-6a745812f898-istio-podinfo\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.228837 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.228796 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/052a18a8-5808-481b-a2db-6a745812f898-istio-token\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.228970 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.228888 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhdch\" (UniqueName: \"kubernetes.io/projected/052a18a8-5808-481b-a2db-6a745812f898-kube-api-access-mhdch\") pod \"e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.240103 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.240073 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw"] Oct 29 23:30:54.242724 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.242692 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:30:54.242827 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.242762 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:30:54.242827 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.242791 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:30:54.297067 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.297041 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" event={"ID":"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5","Type":"ContainerStarted","Data":"b7b02fd8282e44eff4cdf07e1febd2b583ae8b71ccd5cfd10ffa034e73796183"} Oct 29 23:30:54.379104 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.379075 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:54.496013 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.495966 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:54.723529 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:30:54.723485 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod052a18a8_5808_481b_a2db_6a745812f898.slice/crio-7fdcc2b41cdf9f88e8f8e304d02ee4a06a46cbed89c7b1273b8c2a84a0d7d84f WatchSource:0}: Error finding container 7fdcc2b41cdf9f88e8f8e304d02ee4a06a46cbed89c7b1273b8c2a84a0d7d84f: Status 404 returned error can't find the container with id 7fdcc2b41cdf9f88e8f8e304d02ee4a06a46cbed89c7b1273b8c2a84a0d7d84f Oct 29 23:30:54.723861 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.723638 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm"] Oct 29 23:30:54.726195 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.726154 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:30:54.726344 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.726246 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:30:54.726344 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:54.726330 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:30:55.305679 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:55.305642 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" event={"ID":"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5","Type":"ContainerStarted","Data":"460c0a3d4e898311f617251e4c1726b27b6e10999053c96e8cd742d330d8cbc8"} Oct 29 23:30:55.307374 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:55.307346 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" event={"ID":"052a18a8-5808-481b-a2db-6a745812f898","Type":"ContainerStarted","Data":"0bbc39d9a7155f6354a662e8f4f78a93efa290ed53fdb183b157fa3d54df3118"} Oct 29 23:30:55.307512 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:55.307389 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" event={"ID":"052a18a8-5808-481b-a2db-6a745812f898","Type":"ContainerStarted","Data":"7fdcc2b41cdf9f88e8f8e304d02ee4a06a46cbed89c7b1273b8c2a84a0d7d84f"} Oct 29 23:30:55.362248 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:55.362181 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" podStartSLOduration=2.362167099 podStartE2EDuration="2.362167099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:55.36069157 +0000 UTC m=+1082.727439201" watchObservedRunningTime="2026-10-29 23:30:55.362167099 +0000 UTC m=+1082.728914797" Oct 29 23:30:55.379325 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:55.379293 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:30:55.380821 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:55.380796 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 29 23:30:55.380948 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:55.380842 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" podUID="052a18a8-5808-481b-a2db-6a745812f898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 29 23:30:55.412320 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:55.412253 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" podStartSLOduration=2.412241095 podStartE2EDuration="2.412241095s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:30:55.410894746 +0000 UTC m=+1082.777642387" watchObservedRunningTime="2026-10-29 23:30:55.412241095 +0000 UTC m=+1082.778988726" Oct 29 23:30:55.912583 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:55.912549 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:30:55.917393 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:55.917364 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 29 23:30:55.917659 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:55.917629 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" podUID="ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 29 23:30:56.380047 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:56.379958 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 29 23:30:56.380047 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:56.380023 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" podUID="052a18a8-5808-481b-a2db-6a745812f898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 29 23:30:56.656285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:56.656172 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:56.913231 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:56.913149 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 29 23:30:56.913231 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:56.913207 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" podUID="ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 29 23:30:57.382929 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:57.382844 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 29 23:30:57.383211 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:57.383167 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" podUID="052a18a8-5808-481b-a2db-6a745812f898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 29 23:30:57.913116 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:57.913077 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 29 23:30:57.913343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:57.913138 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" podUID="ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 29 23:30:58.381811 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:58.381713 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 29 23:30:58.382081 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:58.382034 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" podUID="052a18a8-5808-481b-a2db-6a745812f898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 29 23:30:58.413772 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:58.413739 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:58.710211 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:58.710174 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:58.924747 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:58.924707 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 29 23:30:58.926472 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:58.925320 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" podUID="ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 29 23:30:59.389292 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:59.385651 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 29 23:30:59.389292 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:59.385710 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" podUID="052a18a8-5808-481b-a2db-6a745812f898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 29 23:30:59.585124 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:59.585071 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:59.730339 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:59.709614 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:30:59.845295 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:59.787872 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:30:59.930608 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:59.927749 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 29 23:30:59.930608 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:30:59.927817 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" podUID="ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 29 23:31:00.384814 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:00.384778 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 29 23:31:00.385140 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:00.385112 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" podUID="052a18a8-5808-481b-a2db-6a745812f898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 29 23:31:00.913004 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:00.912966 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 29 23:31:00.913223 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:00.913037 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" podUID="ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 29 23:31:01.379476 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:01.379400 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 29 23:31:01.379476 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:01.379453 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" podUID="052a18a8-5808-481b-a2db-6a745812f898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 29 23:31:01.879297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:01.878917 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:02.311668 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:02.311635 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:31:02.312098 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:02.312071 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:31:02.313753 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:02.313727 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:31:02.379618 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:02.379582 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 29 23:31:02.380042 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:02.379642 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" podUID="052a18a8-5808-481b-a2db-6a745812f898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 29 23:31:03.384124 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:03.384089 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:31:03.384568 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:03.384515 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:31:03.385370 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:03.385352 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:31:03.748281 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:03.743644 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:05.622288 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:05.615290 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:07.430296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:07.430201 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:08.750855 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:08.750816 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n"] Oct 29 23:31:08.755477 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:08.755451 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:08.794326 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:08.794294 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n"] Oct 29 23:31:08.919906 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:08.919864 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:08.920096 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:08.919957 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:08.920096 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:08.920079 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:08.920233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:08.920107 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt8sd\" (UniqueName: \"kubernetes.io/projected/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-kube-api-access-tt8sd\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:08.920233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:08.920206 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:08.920373 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:08.920289 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:08.920430 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:08.920378 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:08.920430 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:08.920418 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:08.920586 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:08.920567 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.022112 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.022005 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.022447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.022123 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.022447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.022186 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.022447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.022210 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.022447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.022232 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt8sd\" (UniqueName: \"kubernetes.io/projected/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-kube-api-access-tt8sd\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.022447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.022292 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.022447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.022333 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.022447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.022369 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.022447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.022396 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.022863 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.022564 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.023146 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.023122 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.023420 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.023199 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.023556 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.023440 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.023657 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.023635 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.026139 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.026109 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.026613 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.026589 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.050256 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.050223 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.050469 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.050451 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt8sd\" (UniqueName: \"kubernetes.io/projected/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-kube-api-access-tt8sd\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.078539 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.078507 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:09.319385 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.318177 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:09.436975 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.436946 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n"] Oct 29 23:31:09.437442 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:31:09.437414 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb35eb2_2bbf_4250_b7fd_79618c62b17d.slice/crio-b5797a31a1c18bf852d89416a43f411b9f4a81396500e4098e152b286cb3c6ad WatchSource:0}: Error finding container b5797a31a1c18bf852d89416a43f411b9f4a81396500e4098e152b286cb3c6ad: Status 404 returned error can't find the container with id b5797a31a1c18bf852d89416a43f411b9f4a81396500e4098e152b286cb3c6ad Oct 29 23:31:09.440113 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.440083 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:31:09.440185 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.440143 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:31:09.440185 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.440174 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:31:09.483077 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:09.483049 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" event={"ID":"7bb35eb2-2bbf-4250-b7fd-79618c62b17d","Type":"ContainerStarted","Data":"b5797a31a1c18bf852d89416a43f411b9f4a81396500e4098e152b286cb3c6ad"} Oct 29 23:31:10.490446 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:10.490407 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" event={"ID":"7bb35eb2-2bbf-4250-b7fd-79618c62b17d","Type":"ContainerStarted","Data":"34600752b1c85ac2a76e9d81b001691fb4de842f1ed38d966b3618b9cbed8ec8"} Oct 29 23:31:10.570377 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:10.570323 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" podStartSLOduration=2.570303919 podStartE2EDuration="2.570303919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:31:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:31:10.568160132 +0000 UTC m=+1097.934907766" watchObservedRunningTime="2026-10-29 23:31:10.570303919 +0000 UTC m=+1097.937051550" Oct 29 23:31:11.079123 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:11.079091 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:11.080330 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:11.080304 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 29 23:31:11.080462 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:11.080351 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" podUID="7bb35eb2-2bbf-4250-b7fd-79618c62b17d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 29 23:31:11.167282 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:11.166728 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:11.739285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:11.738684 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:12.079622 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:12.079541 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 29 23:31:12.079831 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:12.079808 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" podUID="7bb35eb2-2bbf-4250-b7fd-79618c62b17d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 29 23:31:13.084958 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:13.084425 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 29 23:31:13.084958 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:13.084484 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" podUID="7bb35eb2-2bbf-4250-b7fd-79618c62b17d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 29 23:31:13.230287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:13.227672 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:14.079513 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:14.079480 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 29 23:31:14.079671 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:14.079546 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" podUID="7bb35eb2-2bbf-4250-b7fd-79618c62b17d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 29 23:31:15.079284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:15.077791 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:15.089650 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:15.086700 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 29 23:31:15.089650 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:15.086802 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" podUID="7bb35eb2-2bbf-4250-b7fd-79618c62b17d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 29 23:31:16.079374 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:16.079336 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 29 23:31:16.079750 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:16.079404 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" podUID="7bb35eb2-2bbf-4250-b7fd-79618c62b17d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 29 23:31:17.181033 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:17.180992 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:17.510442 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:17.506649 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:17.510442 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:17.509625 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:17.559692 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:17.557081 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:31:19.393306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:19.391010 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:19.682237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:19.682148 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:21.309189 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:21.309091 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:22.417933 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:31:22.417896 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:31:23.156916 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:23.156863 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:23.327836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:23.327793 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:25.125005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:25.124970 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:27.140987 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:27.140901 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:29.155201 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:29.155158 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:31.243285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:31.237994 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:33.119161 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:33.118869 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:35.062196 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:35.061981 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:36.754523 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:36.754462 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:38.781308 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:38.779545 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:40.543294 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:40.535347 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:42.425285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:42.421515 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:44.269281 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:44.267428 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:44.853395 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:44.853359 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:46.544426 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:46.544381 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:48.155299 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:48.150357 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded" Oct 29 23:31:48.409501 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:48.409389 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded" Oct 29 23:31:48.557624 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:48.557567 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:31:48.578551 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:48.578484 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" podUID="7bb35eb2-2bbf-4250-b7fd-79618c62b17d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:31:49.225696 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:49.225637 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:31:49.460285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:49.451057 5056 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/health\": context deadline exceeded" Oct 29 23:31:49.468780 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:49.468724 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded" Oct 29 23:31:49.529439 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:49.529320 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:31:49.529709 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:49.529665 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" podUID="052a18a8-5808-481b-a2db-6a745812f898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:31:49.976297 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:31:49.975860 5056 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.158s" Oct 29 23:31:49.976297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:49.975951 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:31:49.976297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:49.976040 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:31:49.976297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:49.976070 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:31:49.976297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:49.976098 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:31:50.504370 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:50.504334 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:52.416013 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:31:52.415973 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:31:52.447054 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:52.447005 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:53.793010 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.792977 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6"] Oct 29 23:31:53.798252 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.798232 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.827421 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.827391 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62"] Oct 29 23:31:53.834903 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.834879 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.841273 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.841226 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6"] Oct 29 23:31:53.854091 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854062 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.854246 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854108 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.854246 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854133 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.854246 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854154 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.854246 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854173 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7d71a96-9322-4a8e-9384-6b104852a3b9-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.854246 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854204 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.854246 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854243 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.856944 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854292 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkjt4\" (UniqueName: \"kubernetes.io/projected/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-kube-api-access-pkjt4\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.856944 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854316 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.856944 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854383 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.856944 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854433 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.856944 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854449 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfrjk\" (UniqueName: \"kubernetes.io/projected/b7d71a96-9322-4a8e-9384-6b104852a3b9-kube-api-access-nfrjk\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.856944 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854476 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.856944 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854516 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.856944 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854576 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.856944 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854624 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.856944 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854710 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.856944 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.854771 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.861630 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.861603 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62"] Oct 29 23:31:53.956510 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.956478 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.956672 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.956537 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.956672 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.956596 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.956672 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.956624 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.956672 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.956651 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.956871 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.956673 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.956871 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.956698 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7d71a96-9322-4a8e-9384-6b104852a3b9-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.956871 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.956738 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.956871 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.956781 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.956871 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.956825 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkjt4\" (UniqueName: \"kubernetes.io/projected/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-kube-api-access-pkjt4\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.956871 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.956847 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.957171 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.956895 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.957171 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.956896 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.957171 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.957064 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.957171 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.957143 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-credential-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.957423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.957205 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.957423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.957223 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.957423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.957254 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfrjk\" (UniqueName: \"kubernetes.io/projected/b7d71a96-9322-4a8e-9384-6b104852a3b9-kube-api-access-nfrjk\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.957423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.957309 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.957423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.957342 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-workload-socket\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.957423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.957354 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.957423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.957414 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.957771 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.957465 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.957771 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.957490 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-workload-certs\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.957771 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.957518 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.957771 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.957518 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7d71a96-9322-4a8e-9384-6b104852a3b9-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.958043 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.958022 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-data\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.958440 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.958410 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istiod-ca-cert\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.959836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.959809 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.959915 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.959819 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-podinfo\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.959915 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.959882 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.960010 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.959951 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-envoy\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.984031 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.984003 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:53.984136 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.984067 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-token\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.984186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.984167 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkjt4\" (UniqueName: \"kubernetes.io/projected/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-kube-api-access-pkjt4\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:53.984590 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:53.984572 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfrjk\" (UniqueName: \"kubernetes.io/projected/b7d71a96-9322-4a8e-9384-6b104852a3b9-kube-api-access-nfrjk\") pod \"e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:54.118969 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:54.118876 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:54.160545 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:54.160510 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:54.279529 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:54.270367 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:54.455730 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:54.455666 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6"] Oct 29 23:31:54.464491 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:31:54.464457 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d71a96_9322_4a8e_9384_6b104852a3b9.slice/crio-554bbc393e52c07add2f7be0fbec7c132fef8374a3ed2100c8b6636969d6d988 WatchSource:0}: Error finding container 554bbc393e52c07add2f7be0fbec7c132fef8374a3ed2100c8b6636969d6d988: Status 404 returned error can't find the container with id 554bbc393e52c07add2f7be0fbec7c132fef8374a3ed2100c8b6636969d6d988 Oct 29 23:31:54.467352 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:54.467316 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:31:54.467460 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:54.467386 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:31:54.467460 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:54.467415 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:31:54.540026 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:54.539999 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62"] Oct 29 23:31:54.547126 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:31:54.547099 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9e7834a_0585_49fc_ba76_b68a2a1a48d3.slice/crio-9e54060db461a220a0032a0dc5db10d192600cdee4218ef5e98ccff9f797c0b5 WatchSource:0}: Error finding container 9e54060db461a220a0032a0dc5db10d192600cdee4218ef5e98ccff9f797c0b5: Status 404 returned error can't find the container with id 9e54060db461a220a0032a0dc5db10d192600cdee4218ef5e98ccff9f797c0b5 Oct 29 23:31:54.550318 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:54.550237 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:31:54.550426 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:54.550373 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:31:54.550494 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:54.550424 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:31:55.239656 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:55.239615 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" event={"ID":"b9e7834a-0585-49fc-ba76-b68a2a1a48d3","Type":"ContainerStarted","Data":"fef4f4e901393e0a9e8e4267edaa859da34c236ae50f3cae51b99a3de2e2dde5"} Oct 29 23:31:55.260361 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:55.260305 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" event={"ID":"b9e7834a-0585-49fc-ba76-b68a2a1a48d3","Type":"ContainerStarted","Data":"9e54060db461a220a0032a0dc5db10d192600cdee4218ef5e98ccff9f797c0b5"} Oct 29 23:31:55.282002 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:55.281964 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" event={"ID":"b7d71a96-9322-4a8e-9384-6b104852a3b9","Type":"ContainerStarted","Data":"b6fadb3c2a89c5357276d82735dfc3cfd23d95be54e8c2179d31294eadb1a780"} Oct 29 23:31:55.282184 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:55.282022 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" event={"ID":"b7d71a96-9322-4a8e-9384-6b104852a3b9","Type":"ContainerStarted","Data":"554bbc393e52c07add2f7be0fbec7c132fef8374a3ed2100c8b6636969d6d988"} Oct 29 23:31:55.373313 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:55.368490 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" podStartSLOduration=2.368468925 podStartE2EDuration="2.368468925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:31:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:31:55.364754533 +0000 UTC m=+1142.731502166" watchObservedRunningTime="2026-10-29 23:31:55.368468925 +0000 UTC m=+1142.735216556" Oct 29 23:31:55.471304 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:55.465391 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" podStartSLOduration=2.46537157 podStartE2EDuration="2.46537157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:31:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:31:55.427037002 +0000 UTC m=+1142.793784634" watchObservedRunningTime="2026-10-29 23:31:55.46537157 +0000 UTC m=+1142.832119202" Oct 29 23:31:56.123286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:56.121461 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:31:56.123286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:56.121623 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 29 23:31:56.123286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:56.121685 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" podUID="b7d71a96-9322-4a8e-9384-6b104852a3b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 29 23:31:56.165284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:56.165178 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:31:56.165495 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:56.165325 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body= Oct 29 23:31:56.165495 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:56.165376 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" podUID="b9e7834a-0585-49fc-ba76-b68a2a1a48d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" Oct 29 23:31:56.440283 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:56.429167 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:57.122402 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:57.122361 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 29 23:31:57.145298 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:57.122667 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" podUID="b7d71a96-9322-4a8e-9384-6b104852a3b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 29 23:31:57.162634 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:57.162597 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body= Oct 29 23:31:57.162866 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:57.162839 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" podUID="b9e7834a-0585-49fc-ba76-b68a2a1a48d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" Oct 29 23:31:58.119628 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:58.119589 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 29 23:31:58.120866 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:58.120833 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" podUID="b7d71a96-9322-4a8e-9384-6b104852a3b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 29 23:31:58.163237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:58.163203 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body= Oct 29 23:31:58.163507 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:58.163479 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" podUID="b9e7834a-0585-49fc-ba76-b68a2a1a48d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" Oct 29 23:31:58.649288 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:58.646958 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:59.122346 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.121707 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 29 23:31:59.122346 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.121773 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" podUID="b7d71a96-9322-4a8e-9384-6b104852a3b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 29 23:31:59.174335 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.170439 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body= Oct 29 23:31:59.174335 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.170497 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" podUID="b9e7834a-0585-49fc-ba76-b68a2a1a48d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" Oct 29 23:31:59.589689 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.589460 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n"] Oct 29 23:31:59.595446 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.594506 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:31:59.638307 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.633800 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-2a53c9\"/\"default-dockercfg-8zffd\"" Oct 29 23:31:59.638307 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.634520 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-2a53c9\"/\"kube-root-ca.crt\"" Oct 29 23:31:59.638307 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.634594 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-2a53c9\"/\"openshift-service-ca.crt\"" Oct 29 23:31:59.638307 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.634692 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-2a53c9\"/\"distinct-w1-2a53c9-kserve-self-signed-certs\"" Oct 29 23:31:59.682290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.682242 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n"] Oct 29 23:31:59.766287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.755828 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:31:59.815332 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.801443 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-model-cache\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:31:59.815332 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.801585 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmxsj\" (UniqueName: \"kubernetes.io/projected/a267a091-f7f9-4f00-bf02-5fd668eb27c4-kube-api-access-fmxsj\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:31:59.815332 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.801645 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-dshm\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:31:59.815332 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.801676 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-home\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:31:59.815332 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.801702 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a267a091-f7f9-4f00-bf02-5fd668eb27c4-tls-certs\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:31:59.815332 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.801819 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-tmp-dir\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:31:59.908466 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.908423 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-model-cache\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:31:59.908871 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.908838 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmxsj\" (UniqueName: \"kubernetes.io/projected/a267a091-f7f9-4f00-bf02-5fd668eb27c4-kube-api-access-fmxsj\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:31:59.909046 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.909029 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-dshm\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:31:59.909163 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.909151 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-home\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:31:59.909405 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.909243 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a267a091-f7f9-4f00-bf02-5fd668eb27c4-tls-certs\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:31:59.909632 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.909612 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-tmp-dir\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:31:59.910423 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.910401 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-tmp-dir\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:31:59.919284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.911354 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-model-cache\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:31:59.919777 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.919752 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-home\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:31:59.941286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.934850 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-dshm\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:31:59.991281 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.991103 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmxsj\" (UniqueName: \"kubernetes.io/projected/a267a091-f7f9-4f00-bf02-5fd668eb27c4-kube-api-access-fmxsj\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:32:00.011284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:31:59.991922 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a267a091-f7f9-4f00-bf02-5fd668eb27c4-tls-certs\") pod \"distinct-w1-2a53c9-kserve-78c8d9d945-n529n\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:32:00.122292 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:00.121630 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 29 23:32:00.122292 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:00.121696 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" podUID="b7d71a96-9322-4a8e-9384-6b104852a3b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 29 23:32:00.166947 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:00.166853 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body= Oct 29 23:32:00.167213 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:00.167176 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" podUID="b9e7834a-0585-49fc-ba76-b68a2a1a48d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" Oct 29 23:32:00.269290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:00.254507 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:32:00.679587 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:00.679553 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n"] Oct 29 23:32:01.088091 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:01.087990 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:01.125782 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:01.125737 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 29 23:32:01.126416 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:01.126381 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" podUID="b7d71a96-9322-4a8e-9384-6b104852a3b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 29 23:32:01.163710 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:01.163673 5056 patch_prober.go:29] interesting pod/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body= Oct 29 23:32:01.164007 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:01.163966 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" podUID="b9e7834a-0585-49fc-ba76-b68a2a1a48d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" Oct 29 23:32:01.382805 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:01.382704 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" event={"ID":"a267a091-f7f9-4f00-bf02-5fd668eb27c4","Type":"ContainerStarted","Data":"5cf955376e9f8d7dce5a972989bf9440d75c5a0ee75cc6f64781cd5eddd70a7e"} Oct 29 23:32:01.382805 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:01.382759 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" event={"ID":"a267a091-f7f9-4f00-bf02-5fd668eb27c4","Type":"ContainerStarted","Data":"c3116dbc791fa83cdb6d25b6ffbf1367c31e14b5839232284bee465306e14947"} Oct 29 23:32:02.258704 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:02.258655 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:32:02.645419 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:02.645385 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:32:02.647005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:02.646983 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:32:02.647296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:02.647255 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:32:02.647851 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:02.647831 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:32:02.648836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:02.648816 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:32:02.649887 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:02.649869 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:32:02.711412 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:02.711360 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podStartSLOduration=3.711345129 podStartE2EDuration="3.711345129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:31:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:32:01.430843156 +0000 UTC m=+1148.797590787" watchObservedRunningTime="2026-10-29 23:32:02.711345129 +0000 UTC m=+1150.078092812" Oct 29 23:32:03.057592 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:03.057506 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:05.145354 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:05.144353 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:07.026885 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:07.026688 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:08.894284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:08.889497 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:08.900304 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:08.895011 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz"] Oct 29 23:32:08.909321 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:08.907314 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.034285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.002671 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz"] Oct 29 23:32:09.220470 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.180329 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.220754 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.220730 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.220994 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.220969 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.221127 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.221111 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.221246 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.221233 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.221475 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.221459 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.221737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.221704 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.223245 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.223028 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sgfs\" (UniqueName: \"kubernetes.io/projected/6d1bc223-0aad-4c99-af72-dd9bdd25744f-kube-api-access-2sgfs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.223664 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.223643 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.329688 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.329655 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.330626 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.330591 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2sgfs\" (UniqueName: \"kubernetes.io/projected/6d1bc223-0aad-4c99-af72-dd9bdd25744f-kube-api-access-2sgfs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.331307 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.331283 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.345458 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.330516 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-workload-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.345663 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.331812 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-data\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.345998 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.345975 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.346662 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.346639 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.347808 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.347367 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-workload-certs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.347968 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.347759 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.348089 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.347790 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-credential-socket\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.348242 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.348225 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.348398 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.348382 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.348627 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.348608 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.349270 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.349234 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istiod-ca-cert\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.368348 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.357820 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-podinfo\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.412646 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.412605 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-envoy\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.438286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.413791 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-token\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.438530 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.434011 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sgfs\" (UniqueName: \"kubernetes.io/projected/6d1bc223-0aad-4c99-af72-dd9bdd25744f-kube-api-access-2sgfs\") pod \"e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:09.624339 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:09.620160 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:10.264285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:10.263563 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:32:11.265923 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:11.265800 5056 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:32:11.358520 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:11.358371 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz"] Oct 29 23:32:11.483753 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:11.456532 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded" Oct 29 23:32:11.527199 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:11.527080 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:32:11.527199 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:11.527186 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:32:11.527449 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:11.527234 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:32:12.213385 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:12.213346 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded" Oct 29 23:32:12.303669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:12.303575 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" event={"ID":"6d1bc223-0aad-4c99-af72-dd9bdd25744f","Type":"ContainerStarted","Data":"1f0b85889921eeeb28869e274e03eb2d0b015abf7b1961c25c9f05bb7b05ebe1"} Oct 29 23:32:12.493143 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:12.493097 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:32:12.493978 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:12.493954 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:32:13.250386 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:13.224735 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:32:13.511353 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:13.510354 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded" Oct 29 23:32:13.831571 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:32:13.831473 5056 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.065s" Oct 29 23:32:13.905780 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:13.865463 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:32:14.286572 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:14.286528 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" event={"ID":"6d1bc223-0aad-4c99-af72-dd9bdd25744f","Type":"ContainerStarted","Data":"505ae531fa45e76620059bac8d471838e5109b77a039b9a8b6da01e7ee182194"} Oct 29 23:32:14.411313 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:14.406377 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:32:14.411313 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:14.406441 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" podUID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:32:14.462577 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:14.462521 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" podStartSLOduration=6.462488042 podStartE2EDuration="6.462488042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:32:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:32:14.430473129 +0000 UTC m=+1161.797220765" watchObservedRunningTime="2026-10-29 23:32:14.462488042 +0000 UTC m=+1161.829235673" Oct 29 23:32:14.537496 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:14.537370 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded" Oct 29 23:32:14.538836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:14.538783 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded" Oct 29 23:32:14.643387 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:14.643337 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:14.690412 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:14.690378 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 29 23:32:14.690705 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:14.690664 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" podUID="6d1bc223-0aad-4c99-af72-dd9bdd25744f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 29 23:32:15.555290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:15.554595 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded" Oct 29 23:32:15.555290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:15.554939 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded" Oct 29 23:32:15.606204 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:15.606167 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:32:15.634824 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:15.634792 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 29 23:32:15.635122 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:15.635088 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" podUID="6d1bc223-0aad-4c99-af72-dd9bdd25744f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 29 23:32:15.981019 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:15.980903 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:32:16.281339 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:16.280395 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:16.647481 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:16.647440 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 29 23:32:16.648006 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:16.647506 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" podUID="6d1bc223-0aad-4c99-af72-dd9bdd25744f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 29 23:32:17.626641 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:17.626603 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 29 23:32:17.626930 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:17.626897 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" podUID="6d1bc223-0aad-4c99-af72-dd9bdd25744f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 29 23:32:18.475282 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:18.469930 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:18.625526 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:18.623582 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 29 23:32:18.625526 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:18.623647 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" podUID="6d1bc223-0aad-4c99-af72-dd9bdd25744f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 29 23:32:19.637721 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:19.637680 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 29 23:32:19.638328 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:19.638294 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" podUID="6d1bc223-0aad-4c99-af72-dd9bdd25744f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 29 23:32:19.954141 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:19.954102 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:20.270887 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:20.270802 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:32:20.282335 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:20.282300 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:32:20.622789 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:20.622698 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 29 23:32:20.623066 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:20.623030 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" podUID="6d1bc223-0aad-4c99-af72-dd9bdd25744f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 29 23:32:20.649706 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:20.649671 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:21.629288 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:21.623674 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 29 23:32:21.629288 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:21.623741 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" podUID="6d1bc223-0aad-4c99-af72-dd9bdd25744f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 29 23:32:22.427341 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:32:22.427299 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:32:22.764957 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:22.764822 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:22.899433 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:22.899395 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:22.899820 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:22.899799 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:22.900878 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:22.900855 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:32:24.472658 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.472627 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z"] Oct 29 23:32:24.477189 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.477168 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.569287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.557381 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z"] Oct 29 23:32:24.569287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.566665 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzwvl\" (UniqueName: \"kubernetes.io/projected/e4050977-832e-4b94-9161-5a23e0e27721-kube-api-access-rzwvl\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.569287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.566719 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.569617 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.566802 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4050977-832e-4b94-9161-5a23e0e27721-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.569770 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.569747 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.572577 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.572551 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.572814 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.572781 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4050977-832e-4b94-9161-5a23e0e27721-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.572933 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.572838 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.572933 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.572868 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.572933 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.572904 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4050977-832e-4b94-9161-5a23e0e27721-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.656216 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.656178 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:24.657864 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.657836 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn"] Oct 29 23:32:24.662324 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.662307 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.674549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.674516 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4050977-832e-4b94-9161-5a23e0e27721-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.674651 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.674620 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.674774 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.674760 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.674841 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.674799 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4050977-832e-4b94-9161-5a23e0e27721-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.674841 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.674825 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.674952 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.674857 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.674952 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.674890 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4050977-832e-4b94-9161-5a23e0e27721-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.674952 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.674950 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzwvl\" (UniqueName: \"kubernetes.io/projected/e4050977-832e-4b94-9161-5a23e0e27721-kube-api-access-rzwvl\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.675139 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.674971 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.675445 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.675423 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.675703 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.675681 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.676162 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.676140 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.676843 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.676822 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.677111 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.677077 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4050977-832e-4b94-9161-5a23e0e27721-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.678396 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.678374 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4050977-832e-4b94-9161-5a23e0e27721-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.689733 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.689697 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn"] Oct 29 23:32:24.693340 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.693295 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.701220 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.701196 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4050977-832e-4b94-9161-5a23e0e27721-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.701323 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.701309 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzwvl\" (UniqueName: \"kubernetes.io/projected/e4050977-832e-4b94-9161-5a23e0e27721-kube-api-access-rzwvl\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.775738 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.775662 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.775738 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.775698 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.775952 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.775843 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.776017 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.775964 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.776069 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.776039 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqm72\" (UniqueName: \"kubernetes.io/projected/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-kube-api-access-fqm72\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.776211 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.776195 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.776271 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.776216 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.776326 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.776305 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.776367 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.776360 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.791411 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.791388 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:24.878296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.877730 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.878296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.877866 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.878296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.877942 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqm72\" (UniqueName: \"kubernetes.io/projected/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-kube-api-access-fqm72\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.878296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.878008 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.878296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.878033 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.878296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.878070 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.878296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.878146 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.878296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.878219 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.878296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.878252 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.880110 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.879759 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.880913 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.880838 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.881496 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.881218 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.881496 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.881441 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.881644 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.881597 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.881929 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.881893 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.882828 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.882631 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.906931 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.904479 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.917306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.916372 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqm72\" (UniqueName: \"kubernetes.io/projected/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-kube-api-access-fqm72\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:24.995359 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:24.995327 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:25.130602 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:25.130566 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:32:25.130689 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:25.130636 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:32:25.130689 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:25.130667 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:32:25.134511 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:25.134471 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z"] Oct 29 23:32:25.315919 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:32:25.315865 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce6ba1a5_4a39_4e33_957f_4e8b4c1c0e97.slice/crio-60879c63633c3816588f30b525ba1e70f7c49749c9cc534b0a3fe3f73e6c9eb3 WatchSource:0}: Error finding container 60879c63633c3816588f30b525ba1e70f7c49749c9cc534b0a3fe3f73e6c9eb3: Status 404 returned error can't find the container with id 60879c63633c3816588f30b525ba1e70f7c49749c9cc534b0a3fe3f73e6c9eb3 Oct 29 23:32:25.317525 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:25.317492 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn"] Oct 29 23:32:25.318911 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:25.318874 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:32:25.318973 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:25.318957 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:32:25.319010 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:25.318999 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:32:25.423786 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:25.423747 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" event={"ID":"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97","Type":"ContainerStarted","Data":"2fc6c498419a8c7ac71003feb86893442646362a28a8000ba0b4a6210b18feb8"} Oct 29 23:32:25.423961 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:25.423817 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" event={"ID":"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97","Type":"ContainerStarted","Data":"60879c63633c3816588f30b525ba1e70f7c49749c9cc534b0a3fe3f73e6c9eb3"} Oct 29 23:32:25.427111 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:25.427019 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" event={"ID":"e4050977-832e-4b94-9161-5a23e0e27721","Type":"ContainerStarted","Data":"713c06808d7c6c455b4eccece2a03c38ea39d7cbb3f70835cf8270092baf4c2d"} Oct 29 23:32:25.427111 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:25.427073 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" event={"ID":"e4050977-832e-4b94-9161-5a23e0e27721","Type":"ContainerStarted","Data":"7dfa8b2ef3605803a20642d2a734fa325e540a7464aab3cdbcfb4460e53bdd56"} Oct 29 23:32:25.534307 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:25.531424 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" podStartSLOduration=1.531405244 podStartE2EDuration="1.531405244s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:32:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:32:25.530837909 +0000 UTC m=+1172.897585543" watchObservedRunningTime="2026-10-29 23:32:25.531405244 +0000 UTC m=+1172.898152876" Oct 29 23:32:25.534307 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:25.532252 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" podStartSLOduration=1.532236604 podStartE2EDuration="1.532236604s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:32:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:32:25.478517668 +0000 UTC m=+1172.845265300" watchObservedRunningTime="2026-10-29 23:32:25.532236604 +0000 UTC m=+1172.898984236" Oct 29 23:32:25.791940 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:25.791907 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:25.996520 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:25.996488 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:26.557842 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:26.557806 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:26.800836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:26.800788 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body= Oct 29 23:32:26.801032 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:26.800858 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" podUID="e4050977-832e-4b94-9161-5a23e0e27721" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" Oct 29 23:32:26.998231 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:26.998192 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body= Oct 29 23:32:26.998430 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:26.998279 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" podUID="ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" Oct 29 23:32:27.675738 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:27.675702 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:27.792895 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:27.792865 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body= Oct 29 23:32:27.793074 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:27.792914 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" podUID="e4050977-832e-4b94-9161-5a23e0e27721" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" Oct 29 23:32:27.996680 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:27.996596 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body= Oct 29 23:32:27.996680 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:27.996650 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" podUID="ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" Oct 29 23:32:28.792481 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:28.792436 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body= Oct 29 23:32:28.792887 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:28.792503 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" podUID="e4050977-832e-4b94-9161-5a23e0e27721" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" Oct 29 23:32:29.016284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:29.014287 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:29.067339 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:29.067199 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body= Oct 29 23:32:29.067339 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:29.067256 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" podUID="ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" Oct 29 23:32:29.793310 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:29.793247 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body= Oct 29 23:32:29.793769 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:29.793328 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" podUID="e4050977-832e-4b94-9161-5a23e0e27721" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" Oct 29 23:32:29.997204 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:29.997174 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body= Oct 29 23:32:29.997443 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:29.997420 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" podUID="ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" Oct 29 23:32:30.800892 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:30.800843 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body= Oct 29 23:32:30.801461 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:30.800906 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" podUID="e4050977-832e-4b94-9161-5a23e0e27721" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" Oct 29 23:32:30.995700 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:30.995651 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body= Oct 29 23:32:30.995854 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:30.995709 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" podUID="ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" Oct 29 23:32:31.156354 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:31.141225 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:31.792568 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:31.792527 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body= Oct 29 23:32:31.792745 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:31.792598 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" podUID="e4050977-832e-4b94-9161-5a23e0e27721" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" Oct 29 23:32:31.996153 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:31.996107 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body= Oct 29 23:32:31.996685 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:31.996161 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" podUID="ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" Oct 29 23:32:32.484207 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:32.484178 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:32.796745 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:32.796661 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:32.797100 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:32.797073 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:32.797889 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:32.797870 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:32:33.001056 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:33.001027 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:33.001458 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:33.001363 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:33.001994 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:33.001973 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:32:33.368452 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:33.368405 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:35.529962 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:35.529924 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:36.518186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:36.517372 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded" Oct 29 23:32:36.599466 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:36.599395 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 29 23:32:37.511285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:37.511129 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6"] Oct 29 23:32:37.678288 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:37.675305 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded" Oct 29 23:32:37.736999 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:37.697537 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:32:37.755296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:37.750309 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:38.525282 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:38.524698 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:32:38.551783 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:38.551727 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded" Oct 29 23:32:38.685456 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:38.685367 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:32:38.927951 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:38.927914 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:39.074404 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:39.072749 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:32:39.074404 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:39.072818 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" podUID="6d1bc223-0aad-4c99-af72-dd9bdd25744f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:32:39.498287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:39.498109 5056 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:32:39.631024 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:39.610100 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded" Oct 29 23:32:39.652656 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:39.652611 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded" Oct 29 23:32:39.673290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:39.654122 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:32:39.685413 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:39.685381 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:32:39.791336 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:39.752404 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded" Oct 29 23:32:40.654200 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:40.654155 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded" Oct 29 23:32:40.705290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:40.675524 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded" Oct 29 23:32:40.788287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:40.787446 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:32:41.274391 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:41.274341 5056 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:32:41.699781 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:41.699722 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded" Oct 29 23:32:41.742675 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:41.742621 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:32:42.042336 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:42.039322 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:42.404287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:42.392345 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded" Oct 29 23:32:42.747309 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:42.746395 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:32:42.835415 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:42.835357 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:32:43.313286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:43.311517 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:43.469297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:43.455696 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded" Oct 29 23:32:43.592642 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:43.563496 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:32:43.607487 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:43.607121 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:43.627160 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:43.627130 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:32:43.633477 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:43.633453 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:32:44.854716 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:44.854673 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:45.023294 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:45.020832 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:45.636354 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:45.636309 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:46.190190 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:46.190156 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:47.539656 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:47.539619 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:49.588290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:49.580244 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:50.549816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:50.549782 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6"] Oct 29 23:32:50.717024 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:50.716987 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:51.365277 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:51.365225 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:52.430211 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:32:52.430164 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:32:52.996004 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:52.995972 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:32:53.003089 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:53.003058 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:32:53.602311 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:53.602225 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:53.881339 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:53.877438 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" containerID="cri-o://586eb0433f5c5122cb60a12acdebf8bbba1fa0bf4d8a61bf4a415ea3a994af2f" gracePeriod=44 Oct 29 23:32:54.991493 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:54.855718 5056 generic.go:356] "Generic (PLEG): container finished" podID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerID="586eb0433f5c5122cb60a12acdebf8bbba1fa0bf4d8a61bf4a415ea3a994af2f" exitCode=0 Oct 29 23:32:54.991493 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:54.883493 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:32:55.004290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:54.993470 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-dshm\") pod \"9d0d46e2-581f-4dc5-8888-ee516232efde\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " Oct 29 23:32:55.004290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:54.993537 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d0d46e2-581f-4dc5-8888-ee516232efde-kube-api-access-gnjbv\" (UniqueName: \"kubernetes.io/projected/9d0d46e2-581f-4dc5-8888-ee516232efde-kube-api-access-gnjbv\") pod \"9d0d46e2-581f-4dc5-8888-ee516232efde\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " Oct 29 23:32:55.004290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:54.993596 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9d0d46e2-581f-4dc5-8888-ee516232efde-tls-certs\" (UniqueName: \"kubernetes.io/secret/9d0d46e2-581f-4dc5-8888-ee516232efde-tls-certs\") pod \"9d0d46e2-581f-4dc5-8888-ee516232efde\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " Oct 29 23:32:55.004290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:54.993647 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-model-cache\") pod \"9d0d46e2-581f-4dc5-8888-ee516232efde\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " Oct 29 23:32:55.004290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:54.993721 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-home\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-home\") pod \"9d0d46e2-581f-4dc5-8888-ee516232efde\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " Oct 29 23:32:55.004290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:54.994079 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-tmp-dir\") pod \"9d0d46e2-581f-4dc5-8888-ee516232efde\" (UID: \"9d0d46e2-581f-4dc5-8888-ee516232efde\") " Oct 29 23:32:55.032314 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.026743 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-tmp-dir" pod "9d0d46e2-581f-4dc5-8888-ee516232efde" (UID: "9d0d46e2-581f-4dc5-8888-ee516232efde"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:32:55.032314 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.027327 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-model-cache" pod "9d0d46e2-581f-4dc5-8888-ee516232efde" (UID: "9d0d46e2-581f-4dc5-8888-ee516232efde"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:32:55.032314 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.027519 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-home" pod "9d0d46e2-581f-4dc5-8888-ee516232efde" (UID: "9d0d46e2-581f-4dc5-8888-ee516232efde"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:32:55.129367 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.127884 5056 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-tmp-dir\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:32:55.129367 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.127931 5056 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-model-cache\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:32:55.129367 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.127948 5056 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-home\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:32:55.129367 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.128366 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" event={"ID":"9d0d46e2-581f-4dc5-8888-ee516232efde","Type":"ContainerDied","Data":"586eb0433f5c5122cb60a12acdebf8bbba1fa0bf4d8a61bf4a415ea3a994af2f"} Oct 29 23:32:55.129367 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.128433 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" event={"ID":"9d0d46e2-581f-4dc5-8888-ee516232efde","Type":"ContainerDied","Data":"2ded450a0258c8b6dbede3dc5d883483bb3ca1d0c5aee7f8da7bf9285bdf2f39"} Oct 29 23:32:55.129367 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.128462 5056 scope.go:122] "RemoveContainer" containerID="586eb0433f5c5122cb60a12acdebf8bbba1fa0bf4d8a61bf4a415ea3a994af2f" Oct 29 23:32:55.266520 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.266405 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-dshm" pod "9d0d46e2-581f-4dc5-8888-ee516232efde" (UID: "9d0d46e2-581f-4dc5-8888-ee516232efde"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:32:55.290287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.277437 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d0d46e2-581f-4dc5-8888-ee516232efde-tls-certs" pod "9d0d46e2-581f-4dc5-8888-ee516232efde" (UID: "9d0d46e2-581f-4dc5-8888-ee516232efde"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:32:55.361286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.360596 5056 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d0d46e2-581f-4dc5-8888-ee516232efde-dshm\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:32:55.361286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.360634 5056 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d0d46e2-581f-4dc5-8888-ee516232efde-tls-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:32:55.388291 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.369666 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d0d46e2-581f-4dc5-8888-ee516232efde-kube-api-access-gnjbv" pod "9d0d46e2-581f-4dc5-8888-ee516232efde" (UID: "9d0d46e2-581f-4dc5-8888-ee516232efde"). InnerVolumeSpecName "kube-api-access-gnjbv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:32:55.648493 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.403469 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc"] Oct 29 23:32:55.648493 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.404505 5056 cpu_manager.go:414] "containerMap: removing container" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" Oct 29 23:32:55.648493 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.404526 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" Oct 29 23:32:55.648493 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.404748 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" Oct 29 23:32:55.648493 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.566951 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gnjbv\" (UniqueName: \"kubernetes.io/projected/9d0d46e2-581f-4dc5-8888-ee516232efde-kube-api-access-gnjbv\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:32:55.648493 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.596085 5056 scope.go:122] "RemoveContainer" containerID="586eb0433f5c5122cb60a12acdebf8bbba1fa0bf4d8a61bf4a415ea3a994af2f" Oct 29 23:32:55.648493 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.596874 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" containerName="main" probeResult="failure" output="Get \"https://10.132.0.73:8000/ready\": context deadline exceeded" Oct 29 23:32:55.691087 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.684081 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc"] Oct 29 23:32:55.691087 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.684215 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.703756 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.703713 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.703952 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.703784 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.703952 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.703874 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.703952 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.703944 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.704153 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.703984 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.704153 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.704047 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.704153 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.704095 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.704340 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.704230 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xx85s\" (UniqueName: \"kubernetes.io/projected/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-kube-api-access-xx85s\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.704340 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.704333 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.794288 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:32:55.722863 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"586eb0433f5c5122cb60a12acdebf8bbba1fa0bf4d8a61bf4a415ea3a994af2f\": container with ID starting with 586eb0433f5c5122cb60a12acdebf8bbba1fa0bf4d8a61bf4a415ea3a994af2f not found: ID does not exist" containerID="586eb0433f5c5122cb60a12acdebf8bbba1fa0bf4d8a61bf4a415ea3a994af2f" Oct 29 23:32:55.794288 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.722908 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"586eb0433f5c5122cb60a12acdebf8bbba1fa0bf4d8a61bf4a415ea3a994af2f"} err="failed to get container status \"586eb0433f5c5122cb60a12acdebf8bbba1fa0bf4d8a61bf4a415ea3a994af2f\": rpc error: code = NotFound desc = could not find container \"586eb0433f5c5122cb60a12acdebf8bbba1fa0bf4d8a61bf4a415ea3a994af2f\": container with ID starting with 586eb0433f5c5122cb60a12acdebf8bbba1fa0bf4d8a61bf4a415ea3a994af2f not found: ID does not exist" Oct 29 23:32:55.824116 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.824075 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.829443 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.829412 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.841315 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.833368 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.841612 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.841591 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.842160 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.833919 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-workload-certs\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.842160 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.842013 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-workload-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.842384 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.842365 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.850481 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.850457 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xx85s\" (UniqueName: \"kubernetes.io/projected/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-kube-api-access-xx85s\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.854396 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.846979 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istiod-ca-cert\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.854623 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.854602 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.905649 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.905575 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.906220 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.906198 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.906803 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.906119 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-data\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.963025 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.909602 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-credential-socket\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.963378 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.902999 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-token\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:55.963553 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:55.955973 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xx85s\" (UniqueName: \"kubernetes.io/projected/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-kube-api-access-xx85s\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:56.031580 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:56.031500 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-podinfo\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:56.052630 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:56.052594 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-envoy\") pod \"e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:56.056694 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:56.053234 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6" Oct 29 23:32:56.070188 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:56.070156 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:32:56.312969 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:56.312928 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6"] Oct 29 23:32:56.682287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:56.657021 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-53c44e/distinct-w5-53c44e-kserve-8598c5fc6d-5hkd6"] Oct 29 23:32:56.832391 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:56.824657 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d0d46e2-581f-4dc5-8888-ee516232efde" path="/var/lib/kubelet/pods/9d0d46e2-581f-4dc5-8888-ee516232efde/volumes" Oct 29 23:32:57.610028 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:57.609985 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 29 23:32:58.724285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:58.705331 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc"] Oct 29 23:32:58.724285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:58.718557 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded" Oct 29 23:32:58.781456 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:58.754591 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:32:58.781456 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:58.754663 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" podUID="b46bbd7f-4b97-4b30-b500-1c31f0507898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:32:59.045377 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:59.045248 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:32:59.045664 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:59.045634 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:32:59.045822 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:59.045804 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:32:59.545293 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:59.537898 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:32:59.773512 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:32:59.773423 5056 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.007s" Oct 29 23:32:59.777370 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:59.777331 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" event={"ID":"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850","Type":"ContainerStarted","Data":"ef51d98cb4eb69fb6468a086c421aa586644535b3d5dbf0ced2ba09f7c1365f6"} Oct 29 23:32:59.786411 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:59.782357 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 29 23:32:59.801348 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:59.797921 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:32:59.806295 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:59.803476 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:32:59.806295 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:32:59.803545 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" podUID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:33:00.584407 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:00.584361 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 29 23:33:00.815336 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:00.814589 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": context deadline exceeded" Oct 29 23:33:01.247281 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:01.230731 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:01.276791 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:01.276722 5056 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:33:01.282195 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:01.282169 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:33:02.276908 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:02.276870 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:02.390618 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:02.390571 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" event={"ID":"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850","Type":"ContainerStarted","Data":"9f3d57123fab029cee58404e9328eb28102fb843df4e1b141f42fa2f264f46d3"} Oct 29 23:33:02.454660 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:02.454602 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" podStartSLOduration=7.4545849650000005 podStartE2EDuration="7.454584965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:32:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:33:02.453640713 +0000 UTC m=+1209.820388345" watchObservedRunningTime="2026-10-29 23:33:02.454584965 +0000 UTC m=+1209.821332598" Oct 29 23:33:03.071146 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:03.071107 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:33:03.072702 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:03.072676 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 29 23:33:03.072826 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:03.072724 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" podUID="9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 29 23:33:04.071594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:04.071561 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 29 23:33:04.072011 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:04.071614 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" podUID="9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 29 23:33:04.452566 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:04.452523 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:05.071840 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:05.071791 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 29 23:33:05.074200 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:05.072039 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" podUID="9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 29 23:33:05.573389 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:05.573347 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" containerID="cri-o://78e79340962bcb1ca2c4d7a26f56409579d8790da15f2a0af51ef43a5dfd61ca" gracePeriod=45 Oct 29 23:33:05.708897 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:05.708855 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.75:8000/ready\": dial tcp 10.132.0.75:8000: connect: connection refused" Oct 29 23:33:05.905649 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:05.905622 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:33:06.048286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.042404 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-model-cache\") pod \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " Oct 29 23:33:06.048286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.042470 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-tmp-dir\") pod \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " Oct 29 23:33:06.048286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.042564 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-dshm\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-dshm\") pod \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " Oct 29 23:33:06.048286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.042616 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-home\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-home\") pod \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " Oct 29 23:33:06.048286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.042671 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-kube-api-access-dhh8k\" (UniqueName: \"kubernetes.io/projected/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-kube-api-access-dhh8k\") pod \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " Oct 29 23:33:06.048286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.042718 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-tls-certs\" (UniqueName: \"kubernetes.io/secret/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-tls-certs\") pod \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\" (UID: \"89f122d0-9b8d-40fc-90d5-cb04ea07f31c\") " Oct 29 23:33:06.050281 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.049899 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-tmp-dir" pod "89f122d0-9b8d-40fc-90d5-cb04ea07f31c" (UID: "89f122d0-9b8d-40fc-90d5-cb04ea07f31c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:06.051585 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.051390 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-home" pod "89f122d0-9b8d-40fc-90d5-cb04ea07f31c" (UID: "89f122d0-9b8d-40fc-90d5-cb04ea07f31c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:06.052113 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.052070 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-model-cache" pod "89f122d0-9b8d-40fc-90d5-cb04ea07f31c" (UID: "89f122d0-9b8d-40fc-90d5-cb04ea07f31c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:06.052922 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.052838 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-dshm" pod "89f122d0-9b8d-40fc-90d5-cb04ea07f31c" (UID: "89f122d0-9b8d-40fc-90d5-cb04ea07f31c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:06.053546 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.053524 5056 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-model-cache\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:06.053673 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.053659 5056 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-tmp-dir\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:06.053771 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.053759 5056 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-dshm\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:06.053862 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.053851 5056 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-home\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:06.056178 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.056141 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-tls-certs" pod "89f122d0-9b8d-40fc-90d5-cb04ea07f31c" (UID: "89f122d0-9b8d-40fc-90d5-cb04ea07f31c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:33:06.057661 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.057628 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-kube-api-access-dhh8k" pod "89f122d0-9b8d-40fc-90d5-cb04ea07f31c" (UID: "89f122d0-9b8d-40fc-90d5-cb04ea07f31c"). InnerVolumeSpecName "kube-api-access-dhh8k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:06.071569 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.071543 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 29 23:33:06.071673 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.071596 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" podUID="9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 29 23:33:06.160446 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.158554 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhh8k\" (UniqueName: \"kubernetes.io/projected/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-kube-api-access-dhh8k\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:06.160446 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.158588 5056 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89f122d0-9b8d-40fc-90d5-cb04ea07f31c-tls-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:06.267169 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.267131 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:06.667520 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.667484 5056 generic.go:356] "Generic (PLEG): container finished" podID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerID="78e79340962bcb1ca2c4d7a26f56409579d8790da15f2a0af51ef43a5dfd61ca" exitCode=0 Oct 29 23:33:06.668125 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.668099 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" event={"ID":"89f122d0-9b8d-40fc-90d5-cb04ea07f31c","Type":"ContainerDied","Data":"78e79340962bcb1ca2c4d7a26f56409579d8790da15f2a0af51ef43a5dfd61ca"} Oct 29 23:33:06.668313 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.668289 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" event={"ID":"89f122d0-9b8d-40fc-90d5-cb04ea07f31c","Type":"ContainerDied","Data":"c04868e2d6093b019715b422888d3d025599bda56ff1dd4eff513b6234de2b53"} Oct 29 23:33:06.668606 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.668508 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6" Oct 29 23:33:06.670818 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.668525 5056 scope.go:122] "RemoveContainer" containerID="78e79340962bcb1ca2c4d7a26f56409579d8790da15f2a0af51ef43a5dfd61ca" Oct 29 23:33:06.701418 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.700976 5056 scope.go:122] "RemoveContainer" containerID="78e79340962bcb1ca2c4d7a26f56409579d8790da15f2a0af51ef43a5dfd61ca" Oct 29 23:33:06.701764 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:33:06.701689 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78e79340962bcb1ca2c4d7a26f56409579d8790da15f2a0af51ef43a5dfd61ca\": container with ID starting with 78e79340962bcb1ca2c4d7a26f56409579d8790da15f2a0af51ef43a5dfd61ca not found: ID does not exist" containerID="78e79340962bcb1ca2c4d7a26f56409579d8790da15f2a0af51ef43a5dfd61ca" Oct 29 23:33:06.701837 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.701756 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78e79340962bcb1ca2c4d7a26f56409579d8790da15f2a0af51ef43a5dfd61ca"} err="failed to get container status \"78e79340962bcb1ca2c4d7a26f56409579d8790da15f2a0af51ef43a5dfd61ca\": rpc error: code = NotFound desc = could not find container \"78e79340962bcb1ca2c4d7a26f56409579d8790da15f2a0af51ef43a5dfd61ca\": container with ID starting with 78e79340962bcb1ca2c4d7a26f56409579d8790da15f2a0af51ef43a5dfd61ca not found: ID does not exist" Oct 29 23:33:06.751967 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.751936 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6"] Oct 29 23:33:06.769397 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.769363 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-92f6b3/distinct-2-w3-92f6b3-kserve-6699cb899c-2wgb6"] Oct 29 23:33:06.785883 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.785845 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" path="/var/lib/kubelet/pods/89f122d0-9b8d-40fc-90d5-cb04ea07f31c/volumes" Oct 29 23:33:06.961051 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:06.960960 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:07.071823 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:07.071780 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 29 23:33:07.072011 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:07.071858 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" podUID="9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 29 23:33:08.071939 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:08.071895 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body= Oct 29 23:33:08.072442 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:08.071961 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" podUID="9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" Oct 29 23:33:08.948345 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:08.948160 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:09.076326 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.076256 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:33:09.076785 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.076671 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:33:09.077607 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.077579 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:33:09.664373 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.664333 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2"] Oct 29 23:33:09.665326 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.665296 5056 cpu_manager.go:414] "containerMap: removing container" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" Oct 29 23:33:09.665326 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.665320 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" Oct 29 23:33:09.665559 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.665544 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="89f122d0-9b8d-40fc-90d5-cb04ea07f31c" containerName="main" Oct 29 23:33:09.670625 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.670604 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.679753 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.679725 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww"] Oct 29 23:33:09.684884 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.684859 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.711078 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.710498 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2"] Oct 29 23:33:09.718302 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.715801 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww"] Oct 29 23:33:09.752139 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.751888 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.752139 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.751962 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp4n8\" (UniqueName: \"kubernetes.io/projected/cb9f03d0-0cff-466b-9cc7-180db3286611-kube-api-access-lp4n8\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.752139 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.752033 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.752139 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.752060 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.752139 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.752124 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.752599 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.752157 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.752599 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.752202 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.752599 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.752248 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.752599 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.752293 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.752599 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.752320 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.752599 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.752388 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thjn7\" (UniqueName: \"kubernetes.io/projected/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-kube-api-access-thjn7\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.752599 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.752414 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.752599 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.752485 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.752599 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.752516 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.752599 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.752569 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.753074 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.752654 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb9f03d0-0cff-466b-9cc7-180db3286611-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.753074 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.752679 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.753074 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.752707 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.853716 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.853678 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.853716 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.853721 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.854019 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.853789 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.854019 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.853819 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.854019 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.853874 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.854019 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.853918 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.854019 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.853945 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.854019 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.853970 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.854432 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.854046 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thjn7\" (UniqueName: \"kubernetes.io/projected/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-kube-api-access-thjn7\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.854432 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.854075 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.854432 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.854144 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.854432 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.854159 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.854432 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.854178 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.854432 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.854245 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.854432 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.854407 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.854795 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.854512 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.854795 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.854666 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.854795 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.854755 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.854964 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.854806 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb9f03d0-0cff-466b-9cc7-180db3286611-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.854964 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.854833 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.854964 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.854863 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.855119 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.854987 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.855119 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.855036 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp4n8\" (UniqueName: \"kubernetes.io/projected/cb9f03d0-0cff-466b-9cc7-180db3286611-kube-api-access-lp4n8\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.855225 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.855199 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.855617 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.855589 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.855784 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.855760 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.856063 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.856033 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb9f03d0-0cff-466b-9cc7-180db3286611-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.856253 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.856228 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.856816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.856793 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.857383 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.857364 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.857763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.857739 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.857849 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.857829 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.880233 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.880206 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.880548 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.880524 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.880633 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.880621 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thjn7\" (UniqueName: \"kubernetes.io/projected/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-kube-api-access-thjn7\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:09.881758 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.881738 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp4n8\" (UniqueName: \"kubernetes.io/projected/cb9f03d0-0cff-466b-9cc7-180db3286611-kube-api-access-lp4n8\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:09.990770 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:09.990698 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:10.013511 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:10.013483 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:10.301834 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:10.301761 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:10.321178 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:10.321150 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2"] Oct 29 23:33:10.321550 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:33:10.321528 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb9f03d0_0cff_466b_9cc7_180db3286611.slice/crio-b65827220f754bb18d5abf9fc8f3564e8393ffe6ea742596c130fe066b51c516 WatchSource:0}: Error finding container b65827220f754bb18d5abf9fc8f3564e8393ffe6ea742596c130fe066b51c516: Status 404 returned error can't find the container with id b65827220f754bb18d5abf9fc8f3564e8393ffe6ea742596c130fe066b51c516 Oct 29 23:33:10.329322 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:10.329254 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:33:10.329443 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:10.329424 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:33:10.329492 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:10.329479 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:33:10.367755 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:10.367729 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww"] Oct 29 23:33:10.369113 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:33:10.369074 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8474da7b_b2ae_442f_9d8e_c95b8b2f1c0a.slice/crio-31e44d2b447c8927d4c0a0167327fff0b03b16bf1dd103ae35c3fe09b9220148 WatchSource:0}: Error finding container 31e44d2b447c8927d4c0a0167327fff0b03b16bf1dd103ae35c3fe09b9220148: Status 404 returned error can't find the container with id 31e44d2b447c8927d4c0a0167327fff0b03b16bf1dd103ae35c3fe09b9220148 Oct 29 23:33:10.371766 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:10.371735 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:33:10.371859 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:10.371803 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:33:10.371859 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:10.371832 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:33:10.713609 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:10.713571 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" event={"ID":"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a","Type":"ContainerStarted","Data":"ddab0c9e433ee4cdfacd7c08a145602dbf3b9b903a26295b0ff9a75a79cb38fe"} Oct 29 23:33:10.713820 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:10.713664 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" event={"ID":"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a","Type":"ContainerStarted","Data":"31e44d2b447c8927d4c0a0167327fff0b03b16bf1dd103ae35c3fe09b9220148"} Oct 29 23:33:10.715712 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:10.715684 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" event={"ID":"cb9f03d0-0cff-466b-9cc7-180db3286611","Type":"ContainerStarted","Data":"ce87eb8100b519fc9480b4f436fb93f941b018ecdc9e523746530e209846ca26"} Oct 29 23:33:10.715842 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:10.715730 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" event={"ID":"cb9f03d0-0cff-466b-9cc7-180db3286611","Type":"ContainerStarted","Data":"b65827220f754bb18d5abf9fc8f3564e8393ffe6ea742596c130fe066b51c516"} Oct 29 23:33:10.769019 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:10.768962 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" podStartSLOduration=1.768944318 podStartE2EDuration="1.768944318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:33:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:33:10.762522126 +0000 UTC m=+1218.129269758" watchObservedRunningTime="2026-10-29 23:33:10.768944318 +0000 UTC m=+1218.135691950" Oct 29 23:33:10.949287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:10.948828 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:10.998334 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:10.996401 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:11.016342 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:11.016308 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:11.016601 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:11.016350 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 29 23:33:11.016746 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:11.016723 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" podUID="8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 29 23:33:11.992380 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:11.992348 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 29 23:33:11.992792 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:11.992409 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" podUID="cb9f03d0-0cff-466b-9cc7-180db3286611" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 29 23:33:12.037747 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:12.037710 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 29 23:33:12.037986 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:12.037958 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" podUID="8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 29 23:33:12.134742 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:12.134707 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:12.499161 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:12.499125 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:13.046284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:13.012651 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 29 23:33:13.046284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:13.030340 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" podUID="cb9f03d0-0cff-466b-9cc7-180db3286611" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 29 23:33:13.046284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:13.029816 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 29 23:33:13.046284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:13.030783 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" podUID="8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 29 23:33:13.166292 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:13.160242 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" podStartSLOduration=4.160217823 podStartE2EDuration="4.160217823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:33:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:33:10.813452149 +0000 UTC m=+1218.180199780" watchObservedRunningTime="2026-10-29 23:33:13.160217823 +0000 UTC m=+1220.526965455" Oct 29 23:33:13.195237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:13.190669 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb"] Oct 29 23:33:13.195237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:13.194559 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" podUID="9a2facd5-3558-46ca-a031-37c6874548a2" containerName="maas-api" containerID="cri-o://8afd2854f5f8f4fc20d6a4d9d7e61e267c8d14bee8e821ee897fd3b079332691" gracePeriod=30 Oct 29 23:33:13.359794 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:13.359548 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:13.927676 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:13.927573 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" podUID="9a2facd5-3558-46ca-a031-37c6874548a2" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.59:8443/health\": dial tcp 10.132.0.59:8443: connect: connection refused" Oct 29 23:33:14.052319 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:14.052280 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 29 23:33:14.052884 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:14.052854 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" podUID="8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 29 23:33:14.053944 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:14.053920 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 29 23:33:14.054126 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:14.054106 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" podUID="cb9f03d0-0cff-466b-9cc7-180db3286611" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 29 23:33:14.413201 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:14.413156 5056 generic.go:356] "Generic (PLEG): container finished" podID="9a2facd5-3558-46ca-a031-37c6874548a2" containerID="8afd2854f5f8f4fc20d6a4d9d7e61e267c8d14bee8e821ee897fd3b079332691" exitCode=0 Oct 29 23:33:14.413696 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:14.413668 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" event={"ID":"9a2facd5-3558-46ca-a031-37c6874548a2","Type":"ContainerDied","Data":"8afd2854f5f8f4fc20d6a4d9d7e61e267c8d14bee8e821ee897fd3b079332691"} Oct 29 23:33:14.464519 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:14.464480 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:15.001038 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:14.999004 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 29 23:33:15.001038 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:14.999061 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" podUID="cb9f03d0-0cff-466b-9cc7-180db3286611" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 29 23:33:15.002515 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:15.002417 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:15.014680 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:15.014567 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 29 23:33:15.014680 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:15.014626 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" podUID="8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 29 23:33:15.176339 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:15.175402 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:15.491903 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:15.491844 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:15.751876 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:15.751834 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:15.810287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:15.805471 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:33:15.886644 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:15.886607 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a2facd5-3558-46ca-a031-37c6874548a2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a2facd5-3558-46ca-a031-37c6874548a2-openshift-service-ca\") pod \"9a2facd5-3558-46ca-a031-37c6874548a2\" (UID: \"9a2facd5-3558-46ca-a031-37c6874548a2\") " Oct 29 23:33:15.886916 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:15.886896 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9a2facd5-3558-46ca-a031-37c6874548a2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a2facd5-3558-46ca-a031-37c6874548a2-metrics-certs\") pod \"9a2facd5-3558-46ca-a031-37c6874548a2\" (UID: \"9a2facd5-3558-46ca-a031-37c6874548a2\") " Oct 29 23:33:15.887238 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:15.887222 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9a2facd5-3558-46ca-a031-37c6874548a2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a2facd5-3558-46ca-a031-37c6874548a2-maas-api-tls\") pod \"9a2facd5-3558-46ca-a031-37c6874548a2\" (UID: \"9a2facd5-3558-46ca-a031-37c6874548a2\") " Oct 29 23:33:15.887415 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:15.887400 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a2facd5-3558-46ca-a031-37c6874548a2-kube-api-access-628sz\" (UniqueName: \"kubernetes.io/projected/9a2facd5-3558-46ca-a031-37c6874548a2-kube-api-access-628sz\") pod \"9a2facd5-3558-46ca-a031-37c6874548a2\" (UID: \"9a2facd5-3558-46ca-a031-37c6874548a2\") " Oct 29 23:33:15.897981 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:15.897949 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a2facd5-3558-46ca-a031-37c6874548a2-openshift-service-ca" pod "9a2facd5-3558-46ca-a031-37c6874548a2" (UID: "9a2facd5-3558-46ca-a031-37c6874548a2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:33:15.974289 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:15.970895 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:16.033284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:16.016667 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a2facd5-3558-46ca-a031-37c6874548a2-metrics-certs" pod "9a2facd5-3558-46ca-a031-37c6874548a2" (UID: "9a2facd5-3558-46ca-a031-37c6874548a2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:33:16.033524 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:16.028461 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 29 23:33:16.033665 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:16.033641 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" podUID="cb9f03d0-0cff-466b-9cc7-180db3286611" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 29 23:33:16.055327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:16.029828 5056 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a2facd5-3558-46ca-a031-37c6874548a2-openshift-service-ca\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:16.055563 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:16.028501 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 29 23:33:16.055720 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:16.055692 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" podUID="8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 29 23:33:16.102213 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:16.096314 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a2facd5-3558-46ca-a031-37c6874548a2-maas-api-tls" pod "9a2facd5-3558-46ca-a031-37c6874548a2" (UID: "9a2facd5-3558-46ca-a031-37c6874548a2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:33:16.201400 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:16.157861 5056 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a2facd5-3558-46ca-a031-37c6874548a2-metrics-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:16.256197 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:16.256163 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a2facd5-3558-46ca-a031-37c6874548a2-kube-api-access-628sz" pod "9a2facd5-3558-46ca-a031-37c6874548a2" (UID: "9a2facd5-3558-46ca-a031-37c6874548a2"). InnerVolumeSpecName "kube-api-access-628sz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:16.326065 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:16.321361 5056 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a2facd5-3558-46ca-a031-37c6874548a2-maas-api-tls\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:16.326065 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:16.321412 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-628sz\" (UniqueName: \"kubernetes.io/projected/9a2facd5-3558-46ca-a031-37c6874548a2-kube-api-access-628sz\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:16.736712 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:16.736650 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" event={"ID":"9a2facd5-3558-46ca-a031-37c6874548a2","Type":"ContainerDied","Data":"6d6048a541fcdd895eca1749630946ae9734e212bb9b2f7ed463b8edd7d4c975"} Oct 29 23:33:16.736909 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:16.736731 5056 scope.go:122] "RemoveContainer" containerID="8afd2854f5f8f4fc20d6a4d9d7e61e267c8d14bee8e821ee897fd3b079332691" Oct 29 23:33:16.736985 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:16.736952 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb" Oct 29 23:33:17.047338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:16.993557 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 29 23:33:17.047338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:16.993616 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" podUID="cb9f03d0-0cff-466b-9cc7-180db3286611" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 29 23:33:17.094707 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:17.093808 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 29 23:33:17.094707 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:17.093866 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" podUID="8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 29 23:33:17.111758 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:17.111699 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb"] Oct 29 23:33:17.112029 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:17.112012 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-53c44e-5fd695d785-xfdxb"] Oct 29 23:33:17.403467 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:17.403433 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:17.994284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:17.993664 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 29 23:33:17.994284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:17.993727 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" podUID="cb9f03d0-0cff-466b-9cc7-180db3286611" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 29 23:33:18.018412 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:18.018373 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 29 23:33:18.018711 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:18.018677 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" podUID="8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 29 23:33:18.117648 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:18.117610 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:18.780411 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:18.780374 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a2facd5-3558-46ca-a031-37c6874548a2" path="/var/lib/kubelet/pods/9a2facd5-3558-46ca-a031-37c6874548a2/volumes" Oct 29 23:33:18.808712 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:18.808674 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:18.975861 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:18.975788 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:18.994314 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:18.992824 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body= Oct 29 23:33:18.994314 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:18.992901 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" podUID="cb9f03d0-0cff-466b-9cc7-180db3286611" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" Oct 29 23:33:19.015287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:19.015229 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 29 23:33:19.015610 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:19.015569 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" podUID="8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 29 23:33:19.995061 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:19.995024 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:19.995520 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:19.995458 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:19.996175 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:19.996152 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:33:20.256338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:20.256007 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:20.259293 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:20.257443 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:20.264927 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:20.264902 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:33:21.156284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:21.151472 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:22.690317 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:33:22.690278 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:33:23.145442 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:23.113155 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:23.420043 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:23.419932 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:23.765057 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:23.746497 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:23.778757 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:23.778714 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z"] Oct 29 23:33:23.808636 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:23.807042 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" podUID="e1c93822-3b0a-4f84-a23d-26e04ffc4805" containerName="istio-proxy" containerID="cri-o://6aa79afc6dc04281c395377d0530d243ca28b9f7a0f4be6c059e7964a7c50cf9" gracePeriod=30 Oct 29 23:33:23.911652 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:23.911617 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9"] Oct 29 23:33:23.912130 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:23.912097 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" podUID="d21f739e-967e-4f1f-81e9-679f0c549416" containerName="istio-proxy" containerID="cri-o://6bf3bc17c64d72b298a8641403c5b9bc0f54f68cce7391887d9205090c40876b" gracePeriod=30 Oct 29 23:33:23.914748 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:23.914725 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:24.266660 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.266617 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm"] Oct 29 23:33:24.266932 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.266878 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" podUID="052a18a8-5808-481b-a2db-6a745812f898" containerName="istio-proxy" containerID="cri-o://0bbc39d9a7155f6354a662e8f4f78a93efa290ed53fdb183b157fa3d54df3118" gracePeriod=30 Oct 29 23:33:24.382727 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.369049 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk"] Oct 29 23:33:24.382727 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.370136 5056 cpu_manager.go:414] "containerMap: removing container" podUID="9a2facd5-3558-46ca-a031-37c6874548a2" containerName="maas-api" Oct 29 23:33:24.382727 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.370156 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a2facd5-3558-46ca-a031-37c6874548a2" containerName="maas-api" Oct 29 23:33:24.382727 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.370314 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a2facd5-3558-46ca-a031-37c6874548a2" containerName="maas-api" Oct 29 23:33:24.443254 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.442511 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:24.565206 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.565100 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 29 23:33:24.589334 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.583373 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:33:24.589334 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.583445 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" podUID="29e6194d-41e0-4e53-8698-32ace257af0e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:33:24.610297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.592432 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-ae1784\"/\"kube-root-ca.crt\"" Oct 29 23:33:24.610297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.593062 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-ae1784\"/\"default-dockercfg-95pjs\"" Oct 29 23:33:24.610297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.593149 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-ae1784\"/\"openshift-service-ca.crt\"" Oct 29 23:33:24.610727 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.610662 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-ae1784\"/\"distinct-2-w6-ae1784-kserve-self-signed-certs\"" Oct 29 23:33:24.612798 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.612203 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-model-cache\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:24.612798 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.612336 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-dshm\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:24.612798 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.612417 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da79ef4d-119b-4eed-870b-5e0fb48e60c8-tls-certs\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:24.612798 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.612459 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-home\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:24.612798 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.612496 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-tmp-dir\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:24.612798 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.612530 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g46gd\" (UniqueName: \"kubernetes.io/projected/da79ef4d-119b-4eed-870b-5e0fb48e60c8-kube-api-access-g46gd\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:24.665431 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.662520 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk"] Oct 29 23:33:24.732536 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.732501 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da79ef4d-119b-4eed-870b-5e0fb48e60c8-tls-certs\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:24.758446 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.758409 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-home\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:24.758685 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.758670 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-tmp-dir\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:24.758799 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.758785 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g46gd\" (UniqueName: \"kubernetes.io/projected/da79ef4d-119b-4eed-870b-5e0fb48e60c8-kube-api-access-g46gd\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:24.759208 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.759187 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-model-cache\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:24.759392 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.759377 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-dshm\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:24.786285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.760777 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-tmp-dir\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:24.806757 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.763584 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-home\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:24.806883 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.764165 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-model-cache\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:24.806985 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.794141 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da79ef4d-119b-4eed-870b-5e0fb48e60c8-tls-certs\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:24.863326 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:24.862529 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g46gd\" (UniqueName: \"kubernetes.io/projected/da79ef4d-119b-4eed-870b-5e0fb48e60c8-kube-api-access-g46gd\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:25.016286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:25.015385 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-dshm\") pod \"distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:25.048608 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:25.048567 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:25.522288 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:25.494992 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:26.014806 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:26.014758 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:26.944614 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:26.944573 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:28.401443 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:33:28.401370 5056 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.341s" Oct 29 23:33:28.401443 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:28.401434 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk"] Oct 29 23:33:28.821757 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:28.821719 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" event={"ID":"da79ef4d-119b-4eed-870b-5e0fb48e60c8","Type":"ContainerStarted","Data":"d080530a811ae92158ef19b35bc9948571723cb2f689a288f96ed484e2c398bc"} Oct 29 23:33:29.081948 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:29.081863 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48"] Oct 29 23:33:29.082190 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:29.082123 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" podUID="6d57ed6a-5cef-46d3-8449-2eefc9e26c20" containerName="maas-api" containerID="cri-o://92be338c57782b8c60020785fe4ff4c740f506470c7c42eac0536dff68d72673" gracePeriod=30 Oct 29 23:33:29.190683 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:29.190649 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:29.855285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:29.852421 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" event={"ID":"da79ef4d-119b-4eed-870b-5e0fb48e60c8","Type":"ContainerStarted","Data":"f19887b1c92f9a8231ed13b29c82cf9a1afed849ddd343bcf3cbbc06961bca5e"} Oct 29 23:33:29.855802 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:29.855339 5056 generic.go:356] "Generic (PLEG): container finished" podID="6d57ed6a-5cef-46d3-8449-2eefc9e26c20" containerID="92be338c57782b8c60020785fe4ff4c740f506470c7c42eac0536dff68d72673" exitCode=0 Oct 29 23:33:29.855802 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:29.855516 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" event={"ID":"6d57ed6a-5cef-46d3-8449-2eefc9e26c20","Type":"ContainerDied","Data":"92be338c57782b8c60020785fe4ff4c740f506470c7c42eac0536dff68d72673"} Oct 29 23:33:29.908544 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:29.902328 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" podStartSLOduration=6.902311409 podStartE2EDuration="6.902311409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:33:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:33:29.898857901 +0000 UTC m=+1237.265605534" watchObservedRunningTime="2026-10-29 23:33:29.902311409 +0000 UTC m=+1237.269059040" Oct 29 23:33:30.006717 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.006691 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:33:30.063301 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.058393 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:30.063512 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.063449 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:30.181404 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.181364 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-metrics-certs\") pod \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\" (UID: \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\") " Oct 29 23:33:30.181583 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.181504 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-openshift-service-ca\") pod \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\" (UID: \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\") " Oct 29 23:33:30.181583 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.181567 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-kube-api-access-547fv\" (UniqueName: \"kubernetes.io/projected/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-kube-api-access-547fv\") pod \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\" (UID: \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\") " Oct 29 23:33:30.181972 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.181942 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-maas-api-tls\") pod \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\" (UID: \"6d57ed6a-5cef-46d3-8449-2eefc9e26c20\") " Oct 29 23:33:30.183349 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.183320 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-openshift-service-ca" pod "6d57ed6a-5cef-46d3-8449-2eefc9e26c20" (UID: "6d57ed6a-5cef-46d3-8449-2eefc9e26c20"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:33:30.185024 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.184994 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-maas-api-tls" pod "6d57ed6a-5cef-46d3-8449-2eefc9e26c20" (UID: "6d57ed6a-5cef-46d3-8449-2eefc9e26c20"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:33:30.185549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.185521 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-metrics-certs" pod "6d57ed6a-5cef-46d3-8449-2eefc9e26c20" (UID: "6d57ed6a-5cef-46d3-8449-2eefc9e26c20"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:33:30.186662 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.186641 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-kube-api-access-547fv" pod "6d57ed6a-5cef-46d3-8449-2eefc9e26c20" (UID: "6d57ed6a-5cef-46d3-8449-2eefc9e26c20"). InnerVolumeSpecName "kube-api-access-547fv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:30.236896 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.236860 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:30.286764 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.286716 5056 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-maas-api-tls\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:30.286764 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.286756 5056 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-metrics-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:30.286764 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.286769 5056 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-openshift-service-ca\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:30.287078 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.286783 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-547fv\" (UniqueName: \"kubernetes.io/projected/6d57ed6a-5cef-46d3-8449-2eefc9e26c20-kube-api-access-547fv\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:30.437215 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.437123 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:30.829640 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.829602 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:30.865067 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.865035 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" Oct 29 23:33:30.865491 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.865064 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48" event={"ID":"6d57ed6a-5cef-46d3-8449-2eefc9e26c20","Type":"ContainerDied","Data":"7cefb2468cfeb291cc26b691efb65611d9accdb01f3800b4cd7fa6ba36a84863"} Oct 29 23:33:30.865491 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.865129 5056 scope.go:122] "RemoveContainer" containerID="92be338c57782b8c60020785fe4ff4c740f506470c7c42eac0536dff68d72673" Oct 29 23:33:30.925580 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.925548 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48"] Oct 29 23:33:30.936688 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:30.936655 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-92f6b3-79cbd88fc5-svf48"] Oct 29 23:33:31.123982 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:31.123851 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:33:31.123982 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:31.123910 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" podUID="e1c93822-3b0a-4f84-a23d-26e04ffc4805" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:33:31.759159 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:31.759120 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:33:31.759362 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:31.759185 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" podUID="d21f739e-967e-4f1f-81e9-679f0c549416" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:33:32.480118 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:32.480082 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:32.783271 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:32.783180 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d57ed6a-5cef-46d3-8449-2eefc9e26c20" path="/var/lib/kubelet/pods/6d57ed6a-5cef-46d3-8449-2eefc9e26c20/volumes" Oct 29 23:33:33.263135 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.263097 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:33.387006 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.386974 5056 patch_prober.go:29] interesting pod/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:33:33.387169 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.387054 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" podUID="052a18a8-5808-481b-a2db-6a745812f898" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:33:33.564046 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.564016 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:33:33.649094 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.645137 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-workload-socket\") pod \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " Oct 29 23:33:33.649094 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.645206 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-podinfo\") pod \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " Oct 29 23:33:33.649094 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.645336 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-envoy\") pod \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " Oct 29 23:33:33.649094 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.645367 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-workload-certs\") pod \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " Oct 29 23:33:33.649094 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.645571 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istiod-ca-cert\") pod \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " Oct 29 23:33:33.649094 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.645611 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-credential-socket\") pod \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " Oct 29 23:33:33.649094 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.645687 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-data\") pod \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " Oct 29 23:33:33.649094 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.645761 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1c93822-3b0a-4f84-a23d-26e04ffc4805-kube-api-access-j2lb2\" (UniqueName: \"kubernetes.io/projected/e1c93822-3b0a-4f84-a23d-26e04ffc4805-kube-api-access-j2lb2\") pod \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " Oct 29 23:33:33.649094 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.645844 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-token\" (UniqueName: \"kubernetes.io/projected/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-token\") pod \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\" (UID: \"e1c93822-3b0a-4f84-a23d-26e04ffc4805\") " Oct 29 23:33:33.649094 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.647357 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-workload-certs" pod "e1c93822-3b0a-4f84-a23d-26e04ffc4805" (UID: "e1c93822-3b0a-4f84-a23d-26e04ffc4805"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:33.649094 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.647552 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-workload-socket" pod "e1c93822-3b0a-4f84-a23d-26e04ffc4805" (UID: "e1c93822-3b0a-4f84-a23d-26e04ffc4805"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:33.655097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.655058 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-podinfo" pod "e1c93822-3b0a-4f84-a23d-26e04ffc4805" (UID: "e1c93822-3b0a-4f84-a23d-26e04ffc4805"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:33:33.656080 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.656050 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-token" pod "e1c93822-3b0a-4f84-a23d-26e04ffc4805" (UID: "e1c93822-3b0a-4f84-a23d-26e04ffc4805"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:33.656477 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.656451 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istiod-ca-cert" pod "e1c93822-3b0a-4f84-a23d-26e04ffc4805" (UID: "e1c93822-3b0a-4f84-a23d-26e04ffc4805"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:33:33.656770 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.656747 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-credential-socket" pod "e1c93822-3b0a-4f84-a23d-26e04ffc4805" (UID: "e1c93822-3b0a-4f84-a23d-26e04ffc4805"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:33.656991 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.656957 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-data" pod "e1c93822-3b0a-4f84-a23d-26e04ffc4805" (UID: "e1c93822-3b0a-4f84-a23d-26e04ffc4805"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:33.657193 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.657166 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-envoy" pod "e1c93822-3b0a-4f84-a23d-26e04ffc4805" (UID: "e1c93822-3b0a-4f84-a23d-26e04ffc4805"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:33.665447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.665410 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1c93822-3b0a-4f84-a23d-26e04ffc4805-kube-api-access-j2lb2" pod "e1c93822-3b0a-4f84-a23d-26e04ffc4805" (UID: "e1c93822-3b0a-4f84-a23d-26e04ffc4805"). InnerVolumeSpecName "kube-api-access-j2lb2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:33.748027 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.747994 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:33.748176 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.748030 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:33.748176 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.748046 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:33.748176 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.748061 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2lb2\" (UniqueName: \"kubernetes.io/projected/e1c93822-3b0a-4f84-a23d-26e04ffc4805-kube-api-access-j2lb2\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:33.748176 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.748075 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:33.748176 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.748088 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:33.748176 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.748100 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:33.748176 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.748114 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:33.748176 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.748126 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1c93822-3b0a-4f84-a23d-26e04ffc4805-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:33.866028 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.866003 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:33:33.894644 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.894615 5056 generic.go:356] "Generic (PLEG): container finished" podID="d21f739e-967e-4f1f-81e9-679f0c549416" containerID="6bf3bc17c64d72b298a8641403c5b9bc0f54f68cce7391887d9205090c40876b" exitCode=0 Oct 29 23:33:33.894815 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.894690 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" event={"ID":"d21f739e-967e-4f1f-81e9-679f0c549416","Type":"ContainerDied","Data":"6bf3bc17c64d72b298a8641403c5b9bc0f54f68cce7391887d9205090c40876b"} Oct 29 23:33:33.896515 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.896362 5056 generic.go:356] "Generic (PLEG): container finished" podID="052a18a8-5808-481b-a2db-6a745812f898" containerID="0bbc39d9a7155f6354a662e8f4f78a93efa290ed53fdb183b157fa3d54df3118" exitCode=0 Oct 29 23:33:33.896515 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.896458 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" Oct 29 23:33:33.896515 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.896499 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" event={"ID":"052a18a8-5808-481b-a2db-6a745812f898","Type":"ContainerDied","Data":"0bbc39d9a7155f6354a662e8f4f78a93efa290ed53fdb183b157fa3d54df3118"} Oct 29 23:33:33.896764 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.896540 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm" event={"ID":"052a18a8-5808-481b-a2db-6a745812f898","Type":"ContainerDied","Data":"7fdcc2b41cdf9f88e8f8e304d02ee4a06a46cbed89c7b1273b8c2a84a0d7d84f"} Oct 29 23:33:33.896764 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.896570 5056 scope.go:122] "RemoveContainer" containerID="0bbc39d9a7155f6354a662e8f4f78a93efa290ed53fdb183b157fa3d54df3118" Oct 29 23:33:33.898770 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.898742 5056 generic.go:356] "Generic (PLEG): container finished" podID="e1c93822-3b0a-4f84-a23d-26e04ffc4805" containerID="6aa79afc6dc04281c395377d0530d243ca28b9f7a0f4be6c059e7964a7c50cf9" exitCode=0 Oct 29 23:33:33.898862 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.898808 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" event={"ID":"e1c93822-3b0a-4f84-a23d-26e04ffc4805","Type":"ContainerDied","Data":"6aa79afc6dc04281c395377d0530d243ca28b9f7a0f4be6c059e7964a7c50cf9"} Oct 29 23:33:33.898862 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.898845 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" event={"ID":"e1c93822-3b0a-4f84-a23d-26e04ffc4805","Type":"ContainerDied","Data":"ed90f5bfb935565a2d9da5749446c07b07f2960f5c4d4c549cfb67841c31280c"} Oct 29 23:33:33.898963 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.898913 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z" Oct 29 23:33:33.912061 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.912034 5056 scope.go:122] "RemoveContainer" containerID="0bbc39d9a7155f6354a662e8f4f78a93efa290ed53fdb183b157fa3d54df3118" Oct 29 23:33:33.913482 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:33:33.913452 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bbc39d9a7155f6354a662e8f4f78a93efa290ed53fdb183b157fa3d54df3118\": container with ID starting with 0bbc39d9a7155f6354a662e8f4f78a93efa290ed53fdb183b157fa3d54df3118 not found: ID does not exist" containerID="0bbc39d9a7155f6354a662e8f4f78a93efa290ed53fdb183b157fa3d54df3118" Oct 29 23:33:33.913702 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.913492 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bbc39d9a7155f6354a662e8f4f78a93efa290ed53fdb183b157fa3d54df3118"} err="failed to get container status \"0bbc39d9a7155f6354a662e8f4f78a93efa290ed53fdb183b157fa3d54df3118\": rpc error: code = NotFound desc = could not find container \"0bbc39d9a7155f6354a662e8f4f78a93efa290ed53fdb183b157fa3d54df3118\": container with ID starting with 0bbc39d9a7155f6354a662e8f4f78a93efa290ed53fdb183b157fa3d54df3118 not found: ID does not exist" Oct 29 23:33:33.913702 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.913516 5056 scope.go:122] "RemoveContainer" containerID="6aa79afc6dc04281c395377d0530d243ca28b9f7a0f4be6c059e7964a7c50cf9" Oct 29 23:33:33.940321 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.940291 5056 scope.go:122] "RemoveContainer" containerID="6aa79afc6dc04281c395377d0530d243ca28b9f7a0f4be6c059e7964a7c50cf9" Oct 29 23:33:33.940723 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:33:33.940697 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6aa79afc6dc04281c395377d0530d243ca28b9f7a0f4be6c059e7964a7c50cf9\": container with ID starting with 6aa79afc6dc04281c395377d0530d243ca28b9f7a0f4be6c059e7964a7c50cf9 not found: ID does not exist" containerID="6aa79afc6dc04281c395377d0530d243ca28b9f7a0f4be6c059e7964a7c50cf9" Oct 29 23:33:33.940783 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.940738 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6aa79afc6dc04281c395377d0530d243ca28b9f7a0f4be6c059e7964a7c50cf9"} err="failed to get container status \"6aa79afc6dc04281c395377d0530d243ca28b9f7a0f4be6c059e7964a7c50cf9\": rpc error: code = NotFound desc = could not find container \"6aa79afc6dc04281c395377d0530d243ca28b9f7a0f4be6c059e7964a7c50cf9\": container with ID starting with 6aa79afc6dc04281c395377d0530d243ca28b9f7a0f4be6c059e7964a7c50cf9 not found: ID does not exist" Oct 29 23:33:33.951979 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.951955 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-workload-certs\") pod \"052a18a8-5808-481b-a2db-6a745812f898\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " Oct 29 23:33:33.952091 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.952031 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/052a18a8-5808-481b-a2db-6a745812f898-kube-api-access-mhdch\" (UniqueName: \"kubernetes.io/projected/052a18a8-5808-481b-a2db-6a745812f898-kube-api-access-mhdch\") pod \"052a18a8-5808-481b-a2db-6a745812f898\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " Oct 29 23:33:33.952211 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.952165 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/052a18a8-5808-481b-a2db-6a745812f898-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/052a18a8-5808-481b-a2db-6a745812f898-istio-podinfo\") pod \"052a18a8-5808-481b-a2db-6a745812f898\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " Oct 29 23:33:33.952314 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.952210 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-credential-socket\") pod \"052a18a8-5808-481b-a2db-6a745812f898\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " Oct 29 23:33:33.952314 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.952240 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-workload-socket\") pod \"052a18a8-5808-481b-a2db-6a745812f898\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " Oct 29 23:33:33.952419 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.952402 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-istio-data\") pod \"052a18a8-5808-481b-a2db-6a745812f898\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " Oct 29 23:33:33.952473 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.952450 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-istio-envoy\") pod \"052a18a8-5808-481b-a2db-6a745812f898\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " Oct 29 23:33:33.952577 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.952529 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/052a18a8-5808-481b-a2db-6a745812f898-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/052a18a8-5808-481b-a2db-6a745812f898-istiod-ca-cert\") pod \"052a18a8-5808-481b-a2db-6a745812f898\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " Oct 29 23:33:33.952577 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.952541 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-workload-certs" pod "052a18a8-5808-481b-a2db-6a745812f898" (UID: "052a18a8-5808-481b-a2db-6a745812f898"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:33.952704 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.952580 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/052a18a8-5808-481b-a2db-6a745812f898-istio-token\" (UniqueName: \"kubernetes.io/projected/052a18a8-5808-481b-a2db-6a745812f898-istio-token\") pod \"052a18a8-5808-481b-a2db-6a745812f898\" (UID: \"052a18a8-5808-481b-a2db-6a745812f898\") " Oct 29 23:33:33.952768 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.952745 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-workload-socket" pod "052a18a8-5808-481b-a2db-6a745812f898" (UID: "052a18a8-5808-481b-a2db-6a745812f898"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:33.952914 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.952899 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-credential-socket" pod "052a18a8-5808-481b-a2db-6a745812f898" (UID: "052a18a8-5808-481b-a2db-6a745812f898"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:33.953291 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.953252 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:33.953402 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.953296 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:33.953402 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.953310 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:33.953634 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.953614 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/052a18a8-5808-481b-a2db-6a745812f898-istiod-ca-cert" pod "052a18a8-5808-481b-a2db-6a745812f898" (UID: "052a18a8-5808-481b-a2db-6a745812f898"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:33:33.954155 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.954121 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-istio-data" pod "052a18a8-5808-481b-a2db-6a745812f898" (UID: "052a18a8-5808-481b-a2db-6a745812f898"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:33.955885 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.955764 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/052a18a8-5808-481b-a2db-6a745812f898-istio-token" pod "052a18a8-5808-481b-a2db-6a745812f898" (UID: "052a18a8-5808-481b-a2db-6a745812f898"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:33.955885 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.955855 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/052a18a8-5808-481b-a2db-6a745812f898-kube-api-access-mhdch" pod "052a18a8-5808-481b-a2db-6a745812f898" (UID: "052a18a8-5808-481b-a2db-6a745812f898"). InnerVolumeSpecName "kube-api-access-mhdch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:33.956516 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.956492 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-istio-envoy" pod "052a18a8-5808-481b-a2db-6a745812f898" (UID: "052a18a8-5808-481b-a2db-6a745812f898"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:33.957584 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.957559 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/052a18a8-5808-481b-a2db-6a745812f898-istio-podinfo" pod "052a18a8-5808-481b-a2db-6a745812f898" (UID: "052a18a8-5808-481b-a2db-6a745812f898"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:33:33.968194 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:33.968172 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:33:34.010946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.010913 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z"] Oct 29 23:33:34.015180 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.015151 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-6bp2z"] Oct 29 23:33:34.054669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.054600 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d21f739e-967e-4f1f-81e9-679f0c549416-istio-token\" (UniqueName: \"kubernetes.io/projected/d21f739e-967e-4f1f-81e9-679f0c549416-istio-token\") pod \"d21f739e-967e-4f1f-81e9-679f0c549416\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " Oct 29 23:33:34.054669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.054647 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d21f739e-967e-4f1f-81e9-679f0c549416-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21f739e-967e-4f1f-81e9-679f0c549416-istiod-ca-cert\") pod \"d21f739e-967e-4f1f-81e9-679f0c549416\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " Oct 29 23:33:34.054834 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.054720 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-istio-envoy\") pod \"d21f739e-967e-4f1f-81e9-679f0c549416\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " Oct 29 23:33:34.054834 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.054791 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-istio-data\") pod \"d21f739e-967e-4f1f-81e9-679f0c549416\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " Oct 29 23:33:34.054834 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.054813 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d21f739e-967e-4f1f-81e9-679f0c549416-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21f739e-967e-4f1f-81e9-679f0c549416-istio-podinfo\") pod \"d21f739e-967e-4f1f-81e9-679f0c549416\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " Oct 29 23:33:34.054834 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.054831 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-workload-certs\") pod \"d21f739e-967e-4f1f-81e9-679f0c549416\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " Oct 29 23:33:34.055024 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.054859 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d21f739e-967e-4f1f-81e9-679f0c549416-kube-api-access-pvdkj\" (UniqueName: \"kubernetes.io/projected/d21f739e-967e-4f1f-81e9-679f0c549416-kube-api-access-pvdkj\") pod \"d21f739e-967e-4f1f-81e9-679f0c549416\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " Oct 29 23:33:34.055024 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.054896 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-workload-socket\") pod \"d21f739e-967e-4f1f-81e9-679f0c549416\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " Oct 29 23:33:34.055126 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.055083 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d21f739e-967e-4f1f-81e9-679f0c549416-istiod-ca-cert" pod "d21f739e-967e-4f1f-81e9-679f0c549416" (UID: "d21f739e-967e-4f1f-81e9-679f0c549416"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:33:34.055126 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.055100 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-istio-data" pod "d21f739e-967e-4f1f-81e9-679f0c549416" (UID: "d21f739e-967e-4f1f-81e9-679f0c549416"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:34.055320 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.055285 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-workload-certs" pod "d21f739e-967e-4f1f-81e9-679f0c549416" (UID: "d21f739e-967e-4f1f-81e9-679f0c549416"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:34.055455 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.055382 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-workload-socket" pod "d21f739e-967e-4f1f-81e9-679f0c549416" (UID: "d21f739e-967e-4f1f-81e9-679f0c549416"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:34.055747 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.055725 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-credential-socket\") pod \"d21f739e-967e-4f1f-81e9-679f0c549416\" (UID: \"d21f739e-967e-4f1f-81e9-679f0c549416\") " Oct 29 23:33:34.055950 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.055933 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-credential-socket" pod "d21f739e-967e-4f1f-81e9-679f0c549416" (UID: "d21f739e-967e-4f1f-81e9-679f0c549416"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:34.056339 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.056319 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:34.056339 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.056338 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/052a18a8-5808-481b-a2db-6a745812f898-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:34.056491 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.056351 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:34.056491 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.056365 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:34.056491 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.056377 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:34.056491 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.056384 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/052a18a8-5808-481b-a2db-6a745812f898-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:34.056491 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.056394 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/052a18a8-5808-481b-a2db-6a745812f898-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:34.056491 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.056402 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mhdch\" (UniqueName: \"kubernetes.io/projected/052a18a8-5808-481b-a2db-6a745812f898-kube-api-access-mhdch\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:34.056491 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.056410 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:34.056491 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.056418 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d21f739e-967e-4f1f-81e9-679f0c549416-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:34.056491 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.056426 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/052a18a8-5808-481b-a2db-6a745812f898-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:34.057299 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.057192 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d21f739e-967e-4f1f-81e9-679f0c549416-istio-token" pod "d21f739e-967e-4f1f-81e9-679f0c549416" (UID: "d21f739e-967e-4f1f-81e9-679f0c549416"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:34.057299 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.057241 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d21f739e-967e-4f1f-81e9-679f0c549416-istio-podinfo" pod "d21f739e-967e-4f1f-81e9-679f0c549416" (UID: "d21f739e-967e-4f1f-81e9-679f0c549416"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:33:34.057552 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.057529 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d21f739e-967e-4f1f-81e9-679f0c549416-kube-api-access-pvdkj" pod "d21f739e-967e-4f1f-81e9-679f0c549416" (UID: "d21f739e-967e-4f1f-81e9-679f0c549416"). InnerVolumeSpecName "kube-api-access-pvdkj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:34.057611 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.057535 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-istio-envoy" pod "d21f739e-967e-4f1f-81e9-679f0c549416" (UID: "d21f739e-967e-4f1f-81e9-679f0c549416"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:34.157980 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.157946 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d21f739e-967e-4f1f-81e9-679f0c549416-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:34.157980 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.157975 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d21f739e-967e-4f1f-81e9-679f0c549416-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:34.157980 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.157984 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d21f739e-967e-4f1f-81e9-679f0c549416-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:34.158306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.157993 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvdkj\" (UniqueName: \"kubernetes.io/projected/d21f739e-967e-4f1f-81e9-679f0c549416-kube-api-access-pvdkj\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:34.246504 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.246464 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm"] Oct 29 23:33:34.254037 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.254007 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-v7zvm"] Oct 29 23:33:34.477583 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.477533 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:34.774689 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.774647 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="052a18a8-5808-481b-a2db-6a745812f898" path="/var/lib/kubelet/pods/052a18a8-5808-481b-a2db-6a745812f898/volumes" Oct 29 23:33:34.775503 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.775480 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1c93822-3b0a-4f84-a23d-26e04ffc4805" path="/var/lib/kubelet/pods/e1c93822-3b0a-4f84-a23d-26e04ffc4805/volumes" Oct 29 23:33:34.842424 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:33:34.842389 5056 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21f739e_967e_4f1f_81e9_679f0c549416.slice/crio-b873ee5c22031c577a9e0bb7561598191733c7fa6ad2ef11fd25309a22b2bbe8\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21f739e_967e_4f1f_81e9_679f0c549416.slice\": RecentStats: unable to find data in memory cache]" Oct 29 23:33:34.906552 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.906521 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" Oct 29 23:33:34.906735 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.906528 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9" event={"ID":"d21f739e-967e-4f1f-81e9-679f0c549416","Type":"ContainerDied","Data":"b873ee5c22031c577a9e0bb7561598191733c7fa6ad2ef11fd25309a22b2bbe8"} Oct 29 23:33:34.906735 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.906701 5056 scope.go:122] "RemoveContainer" containerID="6bf3bc17c64d72b298a8641403c5b9bc0f54f68cce7391887d9205090c40876b" Oct 29 23:33:34.965242 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.965204 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9"] Oct 29 23:33:34.978094 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:34.978067 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-53c44e-openshift-default-6954bffb8c-c6gp9"] Oct 29 23:33:35.050499 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:35.050418 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:35.063738 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:35.063712 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:35.929211 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:35.929184 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:36.379338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:36.376319 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:36.772552 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:36.772520 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d21f739e-967e-4f1f-81e9-679f0c549416" path="/var/lib/kubelet/pods/d21f739e-967e-4f1f-81e9-679f0c549416/volumes" Oct 29 23:33:38.286542 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:38.286499 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:38.533691 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:38.533651 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:38.542490 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:38.542415 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n"] Oct 29 23:33:38.543011 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:38.542966 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" podUID="7bb35eb2-2bbf-4250-b7fd-79618c62b17d" containerName="istio-proxy" containerID="cri-o://34600752b1c85ac2a76e9d81b001691fb4de842f1ed38d966b3618b9cbed8ec8" gracePeriod=30 Oct 29 23:33:38.558906 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:38.552396 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr"] Oct 29 23:33:38.558906 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:38.552457 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz"] Oct 29 23:33:38.558906 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:38.552654 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" podUID="6d1bc223-0aad-4c99-af72-dd9bdd25744f" containerName="istio-proxy" containerID="cri-o://505ae531fa45e76620059bac8d471838e5109b77a039b9a8b6da01e7ee182194" gracePeriod=30 Oct 29 23:33:38.558906 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:38.552886 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" podUID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerName="istio-proxy" containerID="cri-o://4e35ad5e6065f0cd661c8dde35970452e12b8de076e52cba24ec6a2d700282df" gracePeriod=30 Oct 29 23:33:38.558906 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:38.556905 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47"] Oct 29 23:33:38.558906 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:38.557068 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" podUID="f0bb96ed-9f9c-46de-b003-fa0e7c199259" containerName="istio-proxy" containerID="cri-o://7eca9ca68238258fa851a056cbdd5758bd71cdd4e79625a21d475afe038d8e1d" gracePeriod=30 Oct 29 23:33:38.781290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:38.780853 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:39.203600 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.203565 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv"] Oct 29 23:33:39.206853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.206825 5056 cpu_manager.go:414] "containerMap: removing container" podUID="6d57ed6a-5cef-46d3-8449-2eefc9e26c20" containerName="maas-api" Oct 29 23:33:39.206853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.206851 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d57ed6a-5cef-46d3-8449-2eefc9e26c20" containerName="maas-api" Oct 29 23:33:39.207038 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.206880 5056 cpu_manager.go:414] "containerMap: removing container" podUID="052a18a8-5808-481b-a2db-6a745812f898" containerName="istio-proxy" Oct 29 23:33:39.207038 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.206887 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="052a18a8-5808-481b-a2db-6a745812f898" containerName="istio-proxy" Oct 29 23:33:39.207038 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.206907 5056 cpu_manager.go:414] "containerMap: removing container" podUID="d21f739e-967e-4f1f-81e9-679f0c549416" containerName="istio-proxy" Oct 29 23:33:39.207038 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.206914 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d21f739e-967e-4f1f-81e9-679f0c549416" containerName="istio-proxy" Oct 29 23:33:39.207038 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.206969 5056 cpu_manager.go:414] "containerMap: removing container" podUID="e1c93822-3b0a-4f84-a23d-26e04ffc4805" containerName="istio-proxy" Oct 29 23:33:39.207038 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.206975 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1c93822-3b0a-4f84-a23d-26e04ffc4805" containerName="istio-proxy" Oct 29 23:33:39.207354 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.207080 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1c93822-3b0a-4f84-a23d-26e04ffc4805" containerName="istio-proxy" Oct 29 23:33:39.207354 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.207090 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d57ed6a-5cef-46d3-8449-2eefc9e26c20" containerName="maas-api" Oct 29 23:33:39.207354 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.207098 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="052a18a8-5808-481b-a2db-6a745812f898" containerName="istio-proxy" Oct 29 23:33:39.207354 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.207104 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="d21f739e-967e-4f1f-81e9-679f0c549416" containerName="istio-proxy" Oct 29 23:33:39.214853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.214826 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.238463 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.238428 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv"] Oct 29 23:33:39.276287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.276215 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/013d55e5-af92-4585-b7c4-33f76c3dabd4-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.276433 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.276294 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.276433 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.276372 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.276433 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.276407 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q57tj\" (UniqueName: \"kubernetes.io/projected/013d55e5-af92-4585-b7c4-33f76c3dabd4-kube-api-access-q57tj\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.276560 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.276538 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.276600 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.276562 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.276635 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.276601 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.276702 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.276687 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.276775 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.276759 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.377928 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.377884 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.377928 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.377927 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.378471 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.377968 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.378471 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.378056 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.378471 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.378127 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.378471 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.378236 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/013d55e5-af92-4585-b7c4-33f76c3dabd4-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.378471 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.378286 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.378471 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.378362 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.378471 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.378396 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q57tj\" (UniqueName: \"kubernetes.io/projected/013d55e5-af92-4585-b7c4-33f76c3dabd4-kube-api-access-q57tj\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.378471 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.378428 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.378471 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.378445 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.378834 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.378544 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.378834 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.378646 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.378903 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.378843 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/013d55e5-af92-4585-b7c4-33f76c3dabd4-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.380392 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.380370 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.380784 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.380767 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.402514 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.402491 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.405798 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.403312 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q57tj\" (UniqueName: \"kubernetes.io/projected/013d55e5-af92-4585-b7c4-33f76c3dabd4-kube-api-access-q57tj\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.538217 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.538125 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:39.785514 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.785482 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z"] Oct 29 23:33:39.791449 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.791382 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.837009 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.836967 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z"] Oct 29 23:33:39.881827 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.881797 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv"] Oct 29 23:33:39.888420 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:33:39.888388 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod013d55e5_af92_4585_b7c4_33f76c3dabd4.slice/crio-51415334dc3241cc7f2df8ab2c2c85b8eee23becd3d12817d16bf602b5efe12d WatchSource:0}: Error finding container 51415334dc3241cc7f2df8ab2c2c85b8eee23becd3d12817d16bf602b5efe12d: Status 404 returned error can't find the container with id 51415334dc3241cc7f2df8ab2c2c85b8eee23becd3d12817d16bf602b5efe12d Oct 29 23:33:39.889084 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.889059 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80bda72d-4e80-497e-befe-514dcaff1e39-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.889190 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.889097 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.889190 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.889138 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msg8f\" (UniqueName: \"kubernetes.io/projected/80bda72d-4e80-497e-befe-514dcaff1e39-kube-api-access-msg8f\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.889190 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.889175 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80bda72d-4e80-497e-befe-514dcaff1e39-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.889375 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.889213 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.889375 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.889254 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.889375 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.889349 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.889535 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.889380 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80bda72d-4e80-497e-befe-514dcaff1e39-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.889535 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.889485 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.890683 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.890642 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:33:39.890764 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.890737 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:33:39.890799 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.890784 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:33:39.961546 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.961513 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" event={"ID":"013d55e5-af92-4585-b7c4-33f76c3dabd4","Type":"ContainerStarted","Data":"51415334dc3241cc7f2df8ab2c2c85b8eee23becd3d12817d16bf602b5efe12d"} Oct 29 23:33:39.991138 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.991103 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msg8f\" (UniqueName: \"kubernetes.io/projected/80bda72d-4e80-497e-befe-514dcaff1e39-kube-api-access-msg8f\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.991343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.991166 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80bda72d-4e80-497e-befe-514dcaff1e39-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.991343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.991206 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.991343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.991244 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.991343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.991306 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.991343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.991334 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80bda72d-4e80-497e-befe-514dcaff1e39-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.991646 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.991427 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.991646 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.991530 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80bda72d-4e80-497e-befe-514dcaff1e39-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.991646 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.991560 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.991802 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.991638 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-credential-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.992063 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.992040 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80bda72d-4e80-497e-befe-514dcaff1e39-istiod-ca-cert\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.992189 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.992171 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-istio-data\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.992328 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.992309 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-workload-certs\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.992381 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.992364 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-workload-socket\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.994191 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.994164 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-istio-envoy\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:39.994447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:39.994424 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80bda72d-4e80-497e-befe-514dcaff1e39-istio-podinfo\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:40.016483 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:40.016448 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msg8f\" (UniqueName: \"kubernetes.io/projected/80bda72d-4e80-497e-befe-514dcaff1e39-kube-api-access-msg8f\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:40.017575 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:40.017548 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80bda72d-4e80-497e-befe-514dcaff1e39-istio-token\") pod \"e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:40.107573 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:40.107476 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:40.465064 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:40.465028 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z"] Oct 29 23:33:40.484296 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:40.479508 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:40.568418 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:33:40.568384 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80bda72d_4e80_497e_befe_514dcaff1e39.slice/crio-691983af408408090f2c023bc3298ca8b2eec012998682d4c0d60665e8c1a3d5 WatchSource:0}: Error finding container 691983af408408090f2c023bc3298ca8b2eec012998682d4c0d60665e8c1a3d5: Status 404 returned error can't find the container with id 691983af408408090f2c023bc3298ca8b2eec012998682d4c0d60665e8c1a3d5 Oct 29 23:33:40.572284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:40.571634 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:33:40.572284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:40.571844 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:33:40.572284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:40.571933 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:33:41.083514 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:41.083476 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" event={"ID":"80bda72d-4e80-497e-befe-514dcaff1e39","Type":"ContainerStarted","Data":"691983af408408090f2c023bc3298ca8b2eec012998682d4c0d60665e8c1a3d5"} Oct 29 23:33:41.106095 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:41.106045 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" event={"ID":"013d55e5-af92-4585-b7c4-33f76c3dabd4","Type":"ContainerStarted","Data":"b97ac2eb5afbe9a86eea78f9fbcaf132147b9152f3b2811fbb16bceec2843965"} Oct 29 23:33:41.214886 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:41.214753 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" podStartSLOduration=2.21473141 podStartE2EDuration="2.21473141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:33:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:33:41.166950522 +0000 UTC m=+1248.533698161" watchObservedRunningTime="2026-10-29 23:33:41.21473141 +0000 UTC m=+1248.581479041" Oct 29 23:33:41.541067 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:41.540974 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:41.545361 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:41.542073 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body= Oct 29 23:33:41.545606 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:41.545580 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" podUID="013d55e5-af92-4585-b7c4-33f76c3dabd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" Oct 29 23:33:41.834072 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:41.833976 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:42.151285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:42.139371 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" event={"ID":"80bda72d-4e80-497e-befe-514dcaff1e39","Type":"ContainerStarted","Data":"70ae58b3d3c9c312981eac19f538081c82c8fa9e43e6382a870fd8a69a040c05"} Oct 29 23:33:42.264533 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:42.264442 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" podStartSLOduration=3.264420829 podStartE2EDuration="3.264420829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:33:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:33:42.212942288 +0000 UTC m=+1249.579689920" watchObservedRunningTime="2026-10-29 23:33:42.264420829 +0000 UTC m=+1249.631168461" Oct 29 23:33:42.541647 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:42.541549 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body= Oct 29 23:33:42.542224 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:42.542179 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" podUID="013d55e5-af92-4585-b7c4-33f76c3dabd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" Oct 29 23:33:43.114280 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:43.114225 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:43.114706 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:43.114681 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body= Oct 29 23:33:43.114864 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:43.114839 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" podUID="80bda72d-4e80-497e-befe-514dcaff1e39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" Oct 29 23:33:43.383350 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:43.381243 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:33:43.383350 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:43.381354 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" podUID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:33:43.643954 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:43.584561 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:33:43.748647 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:43.722338 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body= Oct 29 23:33:43.748647 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:43.722416 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" podUID="013d55e5-af92-4585-b7c4-33f76c3dabd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" Oct 29 23:33:43.867180 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:43.841810 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:33:43.867180 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:43.841908 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" podUID="b46bbd7f-4b97-4b30-b500-1c31f0507898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:33:43.995799 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:43.973416 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" podUID="da79ef4d-119b-4eed-870b-5e0fb48e60c8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 29 23:33:44.234288 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:44.229701 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body= Oct 29 23:33:44.234613 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:44.234572 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" podUID="80bda72d-4e80-497e-befe-514dcaff1e39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" Oct 29 23:33:44.619190 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:44.603665 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 29 23:33:44.636906 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:44.636875 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:33:44.637532 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:44.619253 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body= Oct 29 23:33:44.648981 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:44.637718 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" podUID="013d55e5-af92-4585-b7c4-33f76c3dabd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" Oct 29 23:33:45.166780 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:45.161094 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body= Oct 29 23:33:45.166780 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:45.161145 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" podUID="80bda72d-4e80-497e-befe-514dcaff1e39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" Oct 29 23:33:45.303290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:45.302563 5056 generic.go:356] "Generic (PLEG): container finished" podID="7bb35eb2-2bbf-4250-b7fd-79618c62b17d" containerID="34600752b1c85ac2a76e9d81b001691fb4de842f1ed38d966b3618b9cbed8ec8" exitCode=0 Oct 29 23:33:45.303290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:45.302964 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" event={"ID":"7bb35eb2-2bbf-4250-b7fd-79618c62b17d","Type":"ContainerDied","Data":"34600752b1c85ac2a76e9d81b001691fb4de842f1ed38d966b3618b9cbed8ec8"} Oct 29 23:33:45.321782 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:45.304162 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:45.471460 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:33:45.462555 5056 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb35eb2_2bbf_4250_b7fd_79618c62b17d.slice/crio-34600752b1c85ac2a76e9d81b001691fb4de842f1ed38d966b3618b9cbed8ec8.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c7dc3a_342c_48a9_a569_d9545b1a6653.slice/crio-conmon-4e35ad5e6065f0cd661c8dde35970452e12b8de076e52cba24ec6a2d700282df.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d1bc223_0aad_4c99_af72_dd9bdd25744f.slice/crio-conmon-505ae531fa45e76620059bac8d471838e5109b77a039b9a8b6da01e7ee182194.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0bb96ed_9f9c_46de_b003_fa0e7c199259.slice/crio-7eca9ca68238258fa851a056cbdd5758bd71cdd4e79625a21d475afe038d8e1d.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0bb96ed_9f9c_46de_b003_fa0e7c199259.slice/crio-conmon-7eca9ca68238258fa851a056cbdd5758bd71cdd4e79625a21d475afe038d8e1d.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c7dc3a_342c_48a9_a569_d9545b1a6653.slice/crio-4e35ad5e6065f0cd661c8dde35970452e12b8de076e52cba24ec6a2d700282df.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d1bc223_0aad_4c99_af72_dd9bdd25744f.slice/crio-505ae531fa45e76620059bac8d471838e5109b77a039b9a8b6da01e7ee182194.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb35eb2_2bbf_4250_b7fd_79618c62b17d.slice/crio-conmon-34600752b1c85ac2a76e9d81b001691fb4de842f1ed38d966b3618b9cbed8ec8.scope\": RecentStats: unable to find data in memory cache]" Oct 29 23:33:45.531372 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:33:45.496440 5056 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb35eb2_2bbf_4250_b7fd_79618c62b17d.slice/crio-conmon-34600752b1c85ac2a76e9d81b001691fb4de842f1ed38d966b3618b9cbed8ec8.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d1bc223_0aad_4c99_af72_dd9bdd25744f.slice/crio-505ae531fa45e76620059bac8d471838e5109b77a039b9a8b6da01e7ee182194.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c7dc3a_342c_48a9_a569_d9545b1a6653.slice/crio-conmon-4e35ad5e6065f0cd661c8dde35970452e12b8de076e52cba24ec6a2d700282df.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0bb96ed_9f9c_46de_b003_fa0e7c199259.slice/crio-7eca9ca68238258fa851a056cbdd5758bd71cdd4e79625a21d475afe038d8e1d.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bb35eb2_2bbf_4250_b7fd_79618c62b17d.slice/crio-34600752b1c85ac2a76e9d81b001691fb4de842f1ed38d966b3618b9cbed8ec8.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0bb96ed_9f9c_46de_b003_fa0e7c199259.slice/crio-conmon-7eca9ca68238258fa851a056cbdd5758bd71cdd4e79625a21d475afe038d8e1d.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c7dc3a_342c_48a9_a569_d9545b1a6653.slice/crio-4e35ad5e6065f0cd661c8dde35970452e12b8de076e52cba24ec6a2d700282df.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d1bc223_0aad_4c99_af72_dd9bdd25744f.slice/crio-conmon-505ae531fa45e76620059bac8d471838e5109b77a039b9a8b6da01e7ee182194.scope\": RecentStats: unable to find data in memory cache]" Oct 29 23:33:45.631459 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:45.597390 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body= Oct 29 23:33:45.631459 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:45.597453 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" podUID="013d55e5-af92-4585-b7c4-33f76c3dabd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" Oct 29 23:33:45.718285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:45.690506 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:33:45.728186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:45.728107 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:46.083246 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:46.083148 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 29 23:33:46.083561 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:46.083513 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" podUID="f0bb96ed-9f9c-46de-b003-fa0e7c199259" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 29 23:33:46.084089 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:46.084049 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" podUID="da79ef4d-119b-4eed-870b-5e0fb48e60c8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": context deadline exceeded" Oct 29 23:33:46.136332 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:46.132694 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body= Oct 29 23:33:46.136332 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:46.132783 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" podUID="80bda72d-4e80-497e-befe-514dcaff1e39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" Oct 29 23:33:46.574490 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:46.574451 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body= Oct 29 23:33:46.574775 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:46.574741 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" podUID="013d55e5-af92-4585-b7c4-33f76c3dabd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" Oct 29 23:33:46.681394 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:46.666313 5056 generic.go:356] "Generic (PLEG): container finished" podID="f0bb96ed-9f9c-46de-b003-fa0e7c199259" containerID="7eca9ca68238258fa851a056cbdd5758bd71cdd4e79625a21d475afe038d8e1d" exitCode=0 Oct 29 23:33:46.681394 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:46.666440 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" event={"ID":"f0bb96ed-9f9c-46de-b003-fa0e7c199259","Type":"ContainerDied","Data":"7eca9ca68238258fa851a056cbdd5758bd71cdd4e79625a21d475afe038d8e1d"} Oct 29 23:33:46.746588 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:46.720904 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded" Oct 29 23:33:46.859945 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:46.859841 5056 generic.go:356] "Generic (PLEG): container finished" podID="6d1bc223-0aad-4c99-af72-dd9bdd25744f" containerID="505ae531fa45e76620059bac8d471838e5109b77a039b9a8b6da01e7ee182194" exitCode=0 Oct 29 23:33:46.883783 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:46.878814 5056 generic.go:356] "Generic (PLEG): container finished" podID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerID="4e35ad5e6065f0cd661c8dde35970452e12b8de076e52cba24ec6a2d700282df" exitCode=0 Oct 29 23:33:47.213295 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:47.204401 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" podUID="da79ef4d-119b-4eed-870b-5e0fb48e60c8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:33:47.381188 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:47.381150 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body= Oct 29 23:33:47.381443 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:47.381417 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" podUID="80bda72d-4e80-497e-befe-514dcaff1e39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" Oct 29 23:33:47.534300 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:47.533708 5056 patch_prober.go:29] interesting pod/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 29 23:33:47.534300 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:47.533766 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" podUID="7bb35eb2-2bbf-4250-b7fd-79618c62b17d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 29 23:33:47.782870 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:47.746405 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:47.782870 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:47.746455 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" event={"ID":"6d1bc223-0aad-4c99-af72-dd9bdd25744f","Type":"ContainerDied","Data":"505ae531fa45e76620059bac8d471838e5109b77a039b9a8b6da01e7ee182194"} Oct 29 23:33:47.782870 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:47.746503 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" event={"ID":"40c7dc3a-342c-48a9-a569-d9545b1a6653","Type":"ContainerDied","Data":"4e35ad5e6065f0cd661c8dde35970452e12b8de076e52cba24ec6a2d700282df"} Oct 29 23:33:47.782870 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:47.746549 5056 scope.go:122] "RemoveContainer" containerID="bc858af7fef041166fdb7db0b7e1e4e6354bc8ddba738cc03a7606cfb0ee4eae" Oct 29 23:33:47.782870 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:47.757456 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:33:47.990393 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:47.990349 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-phwkt" podUID="7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23" containerName="proxy" probeResult="failure" output="Get \"https://10.132.0.45:8443/ready\": context deadline exceeded" Oct 29 23:33:47.993308 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:47.992542 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body= Oct 29 23:33:47.993308 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:47.992594 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" podUID="013d55e5-af92-4585-b7c4-33f76c3dabd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" Oct 29 23:33:48.006506 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.005607 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:33:48.096595 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.096561 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:33:48.110588 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.110559 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:33:48.110805 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.110605 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body= Oct 29 23:33:48.110937 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.110917 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" podUID="80bda72d-4e80-497e-befe-514dcaff1e39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" Oct 29 23:33:48.111479 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.111455 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" event={"ID":"40c7dc3a-342c-48a9-a569-d9545b1a6653","Type":"ContainerDied","Data":"b1d6597eb96bf712316f110f5b4b25d1d2e98478ce0960b143b841f7ca28eb49"} Oct 29 23:33:48.183204 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.112927 5056 scope.go:122] "RemoveContainer" containerID="4e35ad5e6065f0cd661c8dde35970452e12b8de076e52cba24ec6a2d700282df" Oct 29 23:33:48.183204 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.124608 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:33:48.200697 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.200673 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" event={"ID":"6d1bc223-0aad-4c99-af72-dd9bdd25744f","Type":"ContainerDied","Data":"1f0b85889921eeeb28869e274e03eb2d0b015abf7b1961c25c9f05bb7b05ebe1"} Oct 29 23:33:48.200895 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.200879 5056 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1f0b85889921eeeb28869e274e03eb2d0b015abf7b1961c25c9f05bb7b05ebe1" Oct 29 23:33:48.211132 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.211069 5056 scope.go:122] "RemoveContainer" containerID="bc858af7fef041166fdb7db0b7e1e4e6354bc8ddba738cc03a7606cfb0ee4eae" Oct 29 23:33:48.211687 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:33:48.211659 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc858af7fef041166fdb7db0b7e1e4e6354bc8ddba738cc03a7606cfb0ee4eae\": container with ID starting with bc858af7fef041166fdb7db0b7e1e4e6354bc8ddba738cc03a7606cfb0ee4eae not found: ID does not exist" containerID="bc858af7fef041166fdb7db0b7e1e4e6354bc8ddba738cc03a7606cfb0ee4eae" Oct 29 23:33:48.211832 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.211692 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc858af7fef041166fdb7db0b7e1e4e6354bc8ddba738cc03a7606cfb0ee4eae"} err="failed to get container status \"bc858af7fef041166fdb7db0b7e1e4e6354bc8ddba738cc03a7606cfb0ee4eae\": rpc error: code = NotFound desc = could not find container \"bc858af7fef041166fdb7db0b7e1e4e6354bc8ddba738cc03a7606cfb0ee4eae\": container with ID starting with bc858af7fef041166fdb7db0b7e1e4e6354bc8ddba738cc03a7606cfb0ee4eae not found: ID does not exist" Oct 29 23:33:48.212886 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.212868 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:33:48.222325 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.213600 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-envoy\") pod \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " Oct 29 23:33:48.222583 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.222561 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-data\") pod \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " Oct 29 23:33:48.231523 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.231445 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-envoy\") pod \"40c7dc3a-342c-48a9-a569-d9545b1a6653\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " Oct 29 23:33:48.231749 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.231713 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40c7dc3a-342c-48a9-a569-d9545b1a6653-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c7dc3a-342c-48a9-a569-d9545b1a6653-istiod-ca-cert\") pod \"40c7dc3a-342c-48a9-a569-d9545b1a6653\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " Oct 29 23:33:48.231933 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.231908 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-token\" (UniqueName: \"kubernetes.io/projected/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-token\") pod \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " Oct 29 23:33:48.246309 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.232070 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-token\" (UniqueName: \"kubernetes.io/projected/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-token\") pod \"40c7dc3a-342c-48a9-a569-d9545b1a6653\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " Oct 29 23:33:48.246680 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.239347 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-data" pod "7bb35eb2-2bbf-4250-b7fd-79618c62b17d" (UID: "7bb35eb2-2bbf-4250-b7fd-79618c62b17d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.246829 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.241905 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-envoy" pod "7bb35eb2-2bbf-4250-b7fd-79618c62b17d" (UID: "7bb35eb2-2bbf-4250-b7fd-79618c62b17d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.246933 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.242300 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40c7dc3a-342c-48a9-a569-d9545b1a6653-istiod-ca-cert" pod "40c7dc3a-342c-48a9-a569-d9545b1a6653" (UID: "40c7dc3a-342c-48a9-a569-d9545b1a6653"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:33:48.247068 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.247049 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-credential-socket\") pod \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " Oct 29 23:33:48.247281 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.247238 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-credential-socket\") pod \"40c7dc3a-342c-48a9-a569-d9545b1a6653\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " Oct 29 23:33:48.247461 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.247442 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istiod-ca-cert\") pod \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " Oct 29 23:33:48.247616 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.247600 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-workload-certs\") pod \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " Oct 29 23:33:48.247819 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.247802 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-workload-socket\") pod \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " Oct 29 23:33:48.248032 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.248014 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-podinfo\") pod \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " Oct 29 23:33:48.248192 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.248175 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40c7dc3a-342c-48a9-a569-d9545b1a6653-kube-api-access-mf4lr\" (UniqueName: \"kubernetes.io/projected/40c7dc3a-342c-48a9-a569-d9545b1a6653-kube-api-access-mf4lr\") pod \"40c7dc3a-342c-48a9-a569-d9545b1a6653\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " Oct 29 23:33:48.248358 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.248343 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-podinfo\") pod \"40c7dc3a-342c-48a9-a569-d9545b1a6653\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " Oct 29 23:33:48.248582 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.248567 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-workload-socket\") pod \"40c7dc3a-342c-48a9-a569-d9545b1a6653\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " Oct 29 23:33:48.248762 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.248735 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-workload-certs\") pod \"40c7dc3a-342c-48a9-a569-d9545b1a6653\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " Oct 29 23:33:48.248945 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.248930 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-kube-api-access-tt8sd\" (UniqueName: \"kubernetes.io/projected/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-kube-api-access-tt8sd\") pod \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\" (UID: \"7bb35eb2-2bbf-4250-b7fd-79618c62b17d\") " Oct 29 23:33:48.249146 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.249128 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-data\") pod \"40c7dc3a-342c-48a9-a569-d9545b1a6653\" (UID: \"40c7dc3a-342c-48a9-a569-d9545b1a6653\") " Oct 29 23:33:48.266524 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.266486 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istiod-ca-cert" pod "7bb35eb2-2bbf-4250-b7fd-79618c62b17d" (UID: "7bb35eb2-2bbf-4250-b7fd-79618c62b17d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:33:48.281284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.269565 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40c7dc3a-342c-48a9-a569-d9545b1a6653-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.281284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.269599 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.281284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.269615 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.281284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.269628 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.281284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.269693 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-credential-socket" pod "7bb35eb2-2bbf-4250-b7fd-79618c62b17d" (UID: "7bb35eb2-2bbf-4250-b7fd-79618c62b17d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.281284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.269765 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-credential-socket" pod "40c7dc3a-342c-48a9-a569-d9545b1a6653" (UID: "40c7dc3a-342c-48a9-a569-d9545b1a6653"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.293278 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.281770 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-workload-socket" pod "40c7dc3a-342c-48a9-a569-d9545b1a6653" (UID: "40c7dc3a-342c-48a9-a569-d9545b1a6653"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.293278 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.272341 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-workload-certs" pod "7bb35eb2-2bbf-4250-b7fd-79618c62b17d" (UID: "7bb35eb2-2bbf-4250-b7fd-79618c62b17d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.293278 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.289671 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-data" pod "40c7dc3a-342c-48a9-a569-d9545b1a6653" (UID: "40c7dc3a-342c-48a9-a569-d9545b1a6653"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.293278 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.290082 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-workload-socket" pod "7bb35eb2-2bbf-4250-b7fd-79618c62b17d" (UID: "7bb35eb2-2bbf-4250-b7fd-79618c62b17d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.297220 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.293551 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-envoy" pod "40c7dc3a-342c-48a9-a569-d9545b1a6653" (UID: "40c7dc3a-342c-48a9-a569-d9545b1a6653"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.300410 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.300385 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-workload-certs" pod "40c7dc3a-342c-48a9-a569-d9545b1a6653" (UID: "40c7dc3a-342c-48a9-a569-d9545b1a6653"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.310778 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.310740 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-token" pod "40c7dc3a-342c-48a9-a569-d9545b1a6653" (UID: "40c7dc3a-342c-48a9-a569-d9545b1a6653"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:48.310986 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.310963 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-podinfo" pod "7bb35eb2-2bbf-4250-b7fd-79618c62b17d" (UID: "7bb35eb2-2bbf-4250-b7fd-79618c62b17d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:33:48.311161 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.311141 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-kube-api-access-tt8sd" pod "7bb35eb2-2bbf-4250-b7fd-79618c62b17d" (UID: "7bb35eb2-2bbf-4250-b7fd-79618c62b17d"). InnerVolumeSpecName "kube-api-access-tt8sd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:48.311295 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.311169 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-token" pod "7bb35eb2-2bbf-4250-b7fd-79618c62b17d" (UID: "7bb35eb2-2bbf-4250-b7fd-79618c62b17d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:48.312672 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.312644 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40c7dc3a-342c-48a9-a569-d9545b1a6653-kube-api-access-mf4lr" pod "40c7dc3a-342c-48a9-a569-d9545b1a6653" (UID: "40c7dc3a-342c-48a9-a569-d9545b1a6653"). InnerVolumeSpecName "kube-api-access-mf4lr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:48.312766 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.312735 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-podinfo" pod "40c7dc3a-342c-48a9-a569-d9545b1a6653" (UID: "40c7dc3a-342c-48a9-a569-d9545b1a6653"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:33:48.387086 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.387043 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-podinfo\") pod \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " Oct 29 23:33:48.387306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.387122 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-credential-socket\") pod \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " Oct 29 23:33:48.387306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.387158 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-workload-socket\") pod \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " Oct 29 23:33:48.387306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.387235 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d1bc223-0aad-4c99-af72-dd9bdd25744f-kube-api-access-2sgfs\" (UniqueName: \"kubernetes.io/projected/6d1bc223-0aad-4c99-af72-dd9bdd25744f-kube-api-access-2sgfs\") pod \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " Oct 29 23:33:48.387306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.387286 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-envoy\") pod \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " Oct 29 23:33:48.387551 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.387397 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-workload-certs\") pod \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " Oct 29 23:33:48.387551 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.387517 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istiod-ca-cert\") pod \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " Oct 29 23:33:48.387658 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.387572 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-data\") pod \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " Oct 29 23:33:48.387708 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.387683 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-token\" (UniqueName: \"kubernetes.io/projected/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-token\") pod \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\" (UID: \"6d1bc223-0aad-4c99-af72-dd9bdd25744f\") " Oct 29 23:33:48.388421 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.388331 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.388421 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.388351 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.388421 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.388360 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.388421 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.388369 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.388421 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.388378 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.388421 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.388388 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.388421 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.388398 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.388421 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.388414 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf4lr\" (UniqueName: \"kubernetes.io/projected/40c7dc3a-342c-48a9-a569-d9545b1a6653-kube-api-access-mf4lr\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.388939 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.388429 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.388939 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.388448 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.388939 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.388460 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.388939 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.388473 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tt8sd\" (UniqueName: \"kubernetes.io/projected/7bb35eb2-2bbf-4250-b7fd-79618c62b17d-kube-api-access-tt8sd\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.388939 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.388487 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.388939 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.388499 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40c7dc3a-342c-48a9-a569-d9545b1a6653-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.388939 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.388573 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-data" pod "6d1bc223-0aad-4c99-af72-dd9bdd25744f" (UID: "6d1bc223-0aad-4c99-af72-dd9bdd25744f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.388939 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.388583 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-credential-socket" pod "6d1bc223-0aad-4c99-af72-dd9bdd25744f" (UID: "6d1bc223-0aad-4c99-af72-dd9bdd25744f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.388939 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.388782 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-workload-certs" pod "6d1bc223-0aad-4c99-af72-dd9bdd25744f" (UID: "6d1bc223-0aad-4c99-af72-dd9bdd25744f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.389194 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.389094 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-workload-socket" pod "6d1bc223-0aad-4c99-af72-dd9bdd25744f" (UID: "6d1bc223-0aad-4c99-af72-dd9bdd25744f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.389194 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.389122 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istiod-ca-cert" pod "6d1bc223-0aad-4c99-af72-dd9bdd25744f" (UID: "6d1bc223-0aad-4c99-af72-dd9bdd25744f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:33:48.391105 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.391076 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-envoy" pod "6d1bc223-0aad-4c99-af72-dd9bdd25744f" (UID: "6d1bc223-0aad-4c99-af72-dd9bdd25744f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.391366 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.391348 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-token" pod "6d1bc223-0aad-4c99-af72-dd9bdd25744f" (UID: "6d1bc223-0aad-4c99-af72-dd9bdd25744f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:48.391562 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.391547 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d1bc223-0aad-4c99-af72-dd9bdd25744f-kube-api-access-2sgfs" pod "6d1bc223-0aad-4c99-af72-dd9bdd25744f" (UID: "6d1bc223-0aad-4c99-af72-dd9bdd25744f"). InnerVolumeSpecName "kube-api-access-2sgfs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:48.391876 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.391853 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-podinfo" pod "6d1bc223-0aad-4c99-af72-dd9bdd25744f" (UID: "6d1bc223-0aad-4c99-af72-dd9bdd25744f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:33:48.398708 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.398648 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:33:48.511379 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.495058 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-token\" (UniqueName: \"kubernetes.io/projected/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-token\") pod \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " Oct 29 23:33:48.511379 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.495107 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-data\") pod \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " Oct 29 23:33:48.511379 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.495144 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-credential-socket\") pod \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " Oct 29 23:33:48.511379 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.495180 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-envoy\") pod \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " Oct 29 23:33:48.511379 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.495297 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istiod-ca-cert\") pod \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " Oct 29 23:33:48.511379 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.495329 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f0bb96ed-9f9c-46de-b003-fa0e7c199259-kube-api-access-x6qp4\" (UniqueName: \"kubernetes.io/projected/f0bb96ed-9f9c-46de-b003-fa0e7c199259-kube-api-access-x6qp4\") pod \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " Oct 29 23:33:48.511379 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.495398 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-workload-socket\") pod \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " Oct 29 23:33:48.513237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.511973 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-workload-certs\") pod \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " Oct 29 23:33:48.513237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.511997 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-credential-socket" pod "f0bb96ed-9f9c-46de-b003-fa0e7c199259" (UID: "f0bb96ed-9f9c-46de-b003-fa0e7c199259"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.513237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.512045 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-podinfo\") pod \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\" (UID: \"f0bb96ed-9f9c-46de-b003-fa0e7c199259\") " Oct 29 23:33:48.513237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.512790 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.513237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.512815 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.513237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.512829 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.513237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.512844 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.513237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.512857 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.513237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.512869 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.513237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.512881 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.513237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.512895 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.513237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.512908 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2sgfs\" (UniqueName: \"kubernetes.io/projected/6d1bc223-0aad-4c99-af72-dd9bdd25744f-kube-api-access-2sgfs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.513237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.512921 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d1bc223-0aad-4c99-af72-dd9bdd25744f-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.513968 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.513307 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-data" pod "f0bb96ed-9f9c-46de-b003-fa0e7c199259" (UID: "f0bb96ed-9f9c-46de-b003-fa0e7c199259"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.525986 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.525880 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-workload-socket" pod "f0bb96ed-9f9c-46de-b003-fa0e7c199259" (UID: "f0bb96ed-9f9c-46de-b003-fa0e7c199259"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.526642 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.526367 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-workload-certs" pod "f0bb96ed-9f9c-46de-b003-fa0e7c199259" (UID: "f0bb96ed-9f9c-46de-b003-fa0e7c199259"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.528421 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.528398 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istiod-ca-cert" pod "f0bb96ed-9f9c-46de-b003-fa0e7c199259" (UID: "f0bb96ed-9f9c-46de-b003-fa0e7c199259"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:33:48.565284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.556432 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-token" pod "f0bb96ed-9f9c-46de-b003-fa0e7c199259" (UID: "f0bb96ed-9f9c-46de-b003-fa0e7c199259"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:48.565284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.556533 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body= Oct 29 23:33:48.565284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.556582 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" podUID="013d55e5-af92-4585-b7c4-33f76c3dabd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" Oct 29 23:33:48.603864 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.603573 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0bb96ed-9f9c-46de-b003-fa0e7c199259-kube-api-access-x6qp4" pod "f0bb96ed-9f9c-46de-b003-fa0e7c199259" (UID: "f0bb96ed-9f9c-46de-b003-fa0e7c199259"). InnerVolumeSpecName "kube-api-access-x6qp4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:48.603864 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.603662 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-podinfo" pod "f0bb96ed-9f9c-46de-b003-fa0e7c199259" (UID: "f0bb96ed-9f9c-46de-b003-fa0e7c199259"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:33:48.603864 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.603718 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-envoy" pod "f0bb96ed-9f9c-46de-b003-fa0e7c199259" (UID: "f0bb96ed-9f9c-46de-b003-fa0e7c199259"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:33:48.648967 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.647187 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.648967 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.647216 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x6qp4\" (UniqueName: \"kubernetes.io/projected/f0bb96ed-9f9c-46de-b003-fa0e7c199259-kube-api-access-x6qp4\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.648967 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.647230 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.648967 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.647243 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.648967 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.647274 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.648967 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.647286 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.648967 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.647298 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.648967 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.647310 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f0bb96ed-9f9c-46de-b003-fa0e7c199259-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:48.894793 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:48.894764 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:49.086373 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.082493 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:49.107801 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.107768 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body= Oct 29 23:33:49.107987 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.107818 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" podUID="80bda72d-4e80-497e-befe-514dcaff1e39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" Oct 29 23:33:49.208358 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.207772 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" Oct 29 23:33:49.208358 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.207789 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n" event={"ID":"7bb35eb2-2bbf-4250-b7fd-79618c62b17d","Type":"ContainerDied","Data":"b5797a31a1c18bf852d89416a43f411b9f4a81396500e4098e152b286cb3c6ad"} Oct 29 23:33:49.208358 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.207875 5056 scope.go:122] "RemoveContainer" containerID="34600752b1c85ac2a76e9d81b001691fb4de842f1ed38d966b3618b9cbed8ec8" Oct 29 23:33:49.211297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.210806 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" Oct 29 23:33:49.211297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.210884 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47" event={"ID":"f0bb96ed-9f9c-46de-b003-fa0e7c199259","Type":"ContainerDied","Data":"e77ff0fe62d1422c484d175ef1e61c986f164367d03497926015a0bcc4564ec4"} Oct 29 23:33:49.212582 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.212563 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr" Oct 29 23:33:49.212783 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.212607 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz" Oct 29 23:33:49.262034 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.262000 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:49.342545 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.342457 5056 scope.go:122] "RemoveContainer" containerID="7eca9ca68238258fa851a056cbdd5758bd71cdd4e79625a21d475afe038d8e1d" Oct 29 23:33:49.424600 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.424566 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n"] Oct 29 23:33:49.428980 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.428949 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-q6q5n"] Oct 29 23:33:49.502787 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.502756 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr"] Oct 29 23:33:49.506236 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.506201 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8r4gr"] Oct 29 23:33:49.557742 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.557701 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body= Oct 29 23:33:49.558037 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.558003 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" podUID="013d55e5-af92-4585-b7c4-33f76c3dabd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" Oct 29 23:33:49.578822 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.578785 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47"] Oct 29 23:33:49.581824 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.581797 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-pmc47"] Oct 29 23:33:49.678401 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.678348 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz"] Oct 29 23:33:49.681627 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:49.681604 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-92f6b3-openshift-default-64f877b587-8rltz"] Oct 29 23:33:50.107900 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:50.107862 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body= Oct 29 23:33:50.108389 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:50.107938 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" podUID="80bda72d-4e80-497e-befe-514dcaff1e39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" Oct 29 23:33:50.539285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:50.538735 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body= Oct 29 23:33:50.539285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:50.538795 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" podUID="013d55e5-af92-4585-b7c4-33f76c3dabd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" Oct 29 23:33:50.784949 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:50.784915 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40c7dc3a-342c-48a9-a569-d9545b1a6653" path="/var/lib/kubelet/pods/40c7dc3a-342c-48a9-a569-d9545b1a6653/volumes" Oct 29 23:33:50.785509 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:50.785492 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d1bc223-0aad-4c99-af72-dd9bdd25744f" path="/var/lib/kubelet/pods/6d1bc223-0aad-4c99-af72-dd9bdd25744f/volumes" Oct 29 23:33:50.785946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:50.785934 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7bb35eb2-2bbf-4250-b7fd-79618c62b17d" path="/var/lib/kubelet/pods/7bb35eb2-2bbf-4250-b7fd-79618c62b17d/volumes" Oct 29 23:33:50.786738 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:50.786723 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f0bb96ed-9f9c-46de-b003-fa0e7c199259" path="/var/lib/kubelet/pods/f0bb96ed-9f9c-46de-b003-fa0e7c199259/volumes" Oct 29 23:33:51.108352 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:51.108314 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body= Oct 29 23:33:51.108738 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:51.108380 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" podUID="80bda72d-4e80-497e-befe-514dcaff1e39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" Oct 29 23:33:51.539695 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:51.539651 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body= Oct 29 23:33:51.541157 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:51.541121 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" podUID="013d55e5-af92-4585-b7c4-33f76c3dabd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" Oct 29 23:33:51.709288 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:51.707000 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:52.108469 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:52.108430 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" start-of-body= Oct 29 23:33:52.108903 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:52.108487 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" podUID="80bda72d-4e80-497e-befe-514dcaff1e39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.90:15021/healthz/ready\": dial tcp 10.132.0.90:15021: connect: connection refused" Oct 29 23:33:52.428975 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:33:52.426417 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:33:52.982864 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:52.982836 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:52.984448 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:52.984424 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:52.985593 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:52.985570 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:33:53.259905 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:53.259812 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:53.261411 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:53.260975 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:53.261889 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:53.261845 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:33:53.749231 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:53.749172 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:54.823581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.823549 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8"] Oct 29 23:33:54.825100 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.825071 5056 cpu_manager.go:414] "containerMap: removing container" podUID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerName="istio-proxy" Oct 29 23:33:54.825240 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.825227 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerName="istio-proxy" Oct 29 23:33:54.825373 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.825359 5056 cpu_manager.go:414] "containerMap: removing container" podUID="6d1bc223-0aad-4c99-af72-dd9bdd25744f" containerName="istio-proxy" Oct 29 23:33:54.825456 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.825443 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d1bc223-0aad-4c99-af72-dd9bdd25744f" containerName="istio-proxy" Oct 29 23:33:54.825617 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.825606 5056 cpu_manager.go:414] "containerMap: removing container" podUID="f0bb96ed-9f9c-46de-b003-fa0e7c199259" containerName="istio-proxy" Oct 29 23:33:54.825706 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.825695 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f0bb96ed-9f9c-46de-b003-fa0e7c199259" containerName="istio-proxy" Oct 29 23:33:54.825792 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.825783 5056 cpu_manager.go:414] "containerMap: removing container" podUID="7bb35eb2-2bbf-4250-b7fd-79618c62b17d" containerName="istio-proxy" Oct 29 23:33:54.825856 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.825847 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7bb35eb2-2bbf-4250-b7fd-79618c62b17d" containerName="istio-proxy" Oct 29 23:33:54.826108 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.826096 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="f0bb96ed-9f9c-46de-b003-fa0e7c199259" containerName="istio-proxy" Oct 29 23:33:54.826193 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.826183 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="7bb35eb2-2bbf-4250-b7fd-79618c62b17d" containerName="istio-proxy" Oct 29 23:33:54.826340 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.826327 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerName="istio-proxy" Oct 29 23:33:54.826456 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.826443 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d1bc223-0aad-4c99-af72-dd9bdd25744f" containerName="istio-proxy" Oct 29 23:33:54.844409 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.844379 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:54.866714 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.866674 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8"] Oct 29 23:33:54.978465 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.978426 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:54.978699 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.978554 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:54.978699 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.978585 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:54.978699 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.978644 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlmr7\" (UniqueName: \"kubernetes.io/projected/6d812120-3b8e-46c4-88d7-febeea4f0670-kube-api-access-dlmr7\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:54.978699 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.978673 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d812120-3b8e-46c4-88d7-febeea4f0670-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:54.979003 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.978802 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:54.979003 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.978844 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:54.979003 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.978886 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:54.979003 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:54.978999 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.032621 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.032588 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:55.083907 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.080753 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.083907 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.080800 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.083907 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.080857 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlmr7\" (UniqueName: \"kubernetes.io/projected/6d812120-3b8e-46c4-88d7-febeea4f0670-kube-api-access-dlmr7\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.083907 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.080885 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d812120-3b8e-46c4-88d7-febeea4f0670-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.092284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.092159 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.092284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.092222 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.092284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.092283 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.092589 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.092407 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.092589 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.092481 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.106339 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.091973 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-workload-certs\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.106525 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.089044 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-credential-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.109779 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.106673 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d812120-3b8e-46c4-88d7-febeea4f0670-istiod-ca-cert\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.109779 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.107244 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-data\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.109779 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.107554 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-workload-socket\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.132524 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.132446 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlmr7\" (UniqueName: \"kubernetes.io/projected/6d812120-3b8e-46c4-88d7-febeea4f0670-kube-api-access-dlmr7\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.141028 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.140955 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-podinfo\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.152158 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.150427 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-token\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.152158 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.150660 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-envoy\") pod \"e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.195254 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.195116 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:55.195976 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.195663 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:55.525356 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:33:55.525314 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d812120_3b8e_46c4_88d7_febeea4f0670.slice/crio-ae9e61cae56ecfdc115e7eff3e934c53b5cee37bf1deba4a74ab21b17ed71ea2 WatchSource:0}: Error finding container ae9e61cae56ecfdc115e7eff3e934c53b5cee37bf1deba4a74ab21b17ed71ea2: Status 404 returned error can't find the container with id ae9e61cae56ecfdc115e7eff3e934c53b5cee37bf1deba4a74ab21b17ed71ea2 Oct 29 23:33:55.525492 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.525467 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8"] Oct 29 23:33:55.527737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.527706 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:33:55.527809 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.527767 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:33:55.527809 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.527796 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:33:55.743707 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.743668 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:55.904949 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:55.904909 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:56.423315 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:56.419447 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" event={"ID":"6d812120-3b8e-46c4-88d7-febeea4f0670","Type":"ContainerStarted","Data":"75b6ba10c4fdca6c2c65f261457bd420d5bb6d07574ce9e282230d799c581c02"} Oct 29 23:33:56.423315 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:56.419517 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" event={"ID":"6d812120-3b8e-46c4-88d7-febeea4f0670","Type":"ContainerStarted","Data":"ae9e61cae56ecfdc115e7eff3e934c53b5cee37bf1deba4a74ab21b17ed71ea2"} Oct 29 23:33:56.501433 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:56.501367 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" podStartSLOduration=2.501347592 podStartE2EDuration="2.501347592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:33:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:33:56.497302084 +0000 UTC m=+1263.864049712" watchObservedRunningTime="2026-10-29 23:33:56.501347592 +0000 UTC m=+1263.868095417" Oct 29 23:33:57.198438 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:57.198406 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:33:57.200025 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:57.199996 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 29 23:33:57.200160 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:57.200046 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" podUID="6d812120-3b8e-46c4-88d7-febeea4f0670" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 29 23:33:57.211121 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:57.211093 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs"] Oct 29 23:33:57.211325 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:57.211290 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" podUID="d3c8f079-e7ec-4d31-9477-e9edc61446db" containerName="maas-api" containerID="cri-o://4aacefbad2fcfafed218ea646dce6127541da2e325588ed467ecfb0d44bacc44" gracePeriod=30 Oct 29 23:33:57.348333 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:57.331081 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:57.596285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:57.595603 5056 generic.go:356] "Generic (PLEG): container finished" podID="d3c8f079-e7ec-4d31-9477-e9edc61446db" containerID="4aacefbad2fcfafed218ea646dce6127541da2e325588ed467ecfb0d44bacc44" exitCode=0 Oct 29 23:33:57.608294 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:57.607115 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" event={"ID":"d3c8f079-e7ec-4d31-9477-e9edc61446db","Type":"ContainerDied","Data":"4aacefbad2fcfafed218ea646dce6127541da2e325588ed467ecfb0d44bacc44"} Oct 29 23:33:58.167414 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.155468 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:33:58.201751 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.199117 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3c8f079-e7ec-4d31-9477-e9edc61446db-kube-api-access-jnhch\" (UniqueName: \"kubernetes.io/projected/d3c8f079-e7ec-4d31-9477-e9edc61446db-kube-api-access-jnhch\") pod \"d3c8f079-e7ec-4d31-9477-e9edc61446db\" (UID: \"d3c8f079-e7ec-4d31-9477-e9edc61446db\") " Oct 29 23:33:58.201751 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.199194 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3c8f079-e7ec-4d31-9477-e9edc61446db-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3c8f079-e7ec-4d31-9477-e9edc61446db-maas-api-tls\") pod \"d3c8f079-e7ec-4d31-9477-e9edc61446db\" (UID: \"d3c8f079-e7ec-4d31-9477-e9edc61446db\") " Oct 29 23:33:58.201751 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.199302 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3c8f079-e7ec-4d31-9477-e9edc61446db-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c8f079-e7ec-4d31-9477-e9edc61446db-metrics-certs\") pod \"d3c8f079-e7ec-4d31-9477-e9edc61446db\" (UID: \"d3c8f079-e7ec-4d31-9477-e9edc61446db\") " Oct 29 23:33:58.201751 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.199503 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d3c8f079-e7ec-4d31-9477-e9edc61446db-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c8f079-e7ec-4d31-9477-e9edc61446db-openshift-service-ca\") pod \"d3c8f079-e7ec-4d31-9477-e9edc61446db\" (UID: \"d3c8f079-e7ec-4d31-9477-e9edc61446db\") " Oct 29 23:33:58.201751 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.200440 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 29 23:33:58.201751 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.200499 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" podUID="6d812120-3b8e-46c4-88d7-febeea4f0670" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 29 23:33:58.201751 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.201021 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3c8f079-e7ec-4d31-9477-e9edc61446db-openshift-service-ca" pod "d3c8f079-e7ec-4d31-9477-e9edc61446db" (UID: "d3c8f079-e7ec-4d31-9477-e9edc61446db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:33:58.204493 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.204464 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3c8f079-e7ec-4d31-9477-e9edc61446db-metrics-certs" pod "d3c8f079-e7ec-4d31-9477-e9edc61446db" (UID: "d3c8f079-e7ec-4d31-9477-e9edc61446db"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:33:58.227609 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.205441 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3c8f079-e7ec-4d31-9477-e9edc61446db-maas-api-tls" pod "d3c8f079-e7ec-4d31-9477-e9edc61446db" (UID: "d3c8f079-e7ec-4d31-9477-e9edc61446db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:33:58.227609 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.207570 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3c8f079-e7ec-4d31-9477-e9edc61446db-kube-api-access-jnhch" pod "d3c8f079-e7ec-4d31-9477-e9edc61446db" (UID: "d3c8f079-e7ec-4d31-9477-e9edc61446db"). InnerVolumeSpecName "kube-api-access-jnhch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:33:58.290236 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.290187 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:58.300957 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.300930 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnhch\" (UniqueName: \"kubernetes.io/projected/d3c8f079-e7ec-4d31-9477-e9edc61446db-kube-api-access-jnhch\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:58.300957 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.300956 5056 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3c8f079-e7ec-4d31-9477-e9edc61446db-maas-api-tls\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:58.301186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.300968 5056 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3c8f079-e7ec-4d31-9477-e9edc61446db-metrics-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:58.301186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.300981 5056 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3c8f079-e7ec-4d31-9477-e9edc61446db-openshift-service-ca\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:33:58.505769 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.504210 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:58.650294 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.638162 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" Oct 29 23:33:58.665487 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.665447 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs" event={"ID":"d3c8f079-e7ec-4d31-9477-e9edc61446db","Type":"ContainerDied","Data":"e7904a2616fc3ce511e6cf5d02c4542018cdb12fc0dccd418226b7e4c5114d35"} Oct 29 23:33:58.665786 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.665735 5056 scope.go:122] "RemoveContainer" containerID="4aacefbad2fcfafed218ea646dce6127541da2e325588ed467ecfb0d44bacc44" Oct 29 23:33:58.667123 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.667079 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:58.810551 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.810520 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs"] Oct 29 23:33:58.810759 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:58.810746 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-b978a0-555b475df5-2dgqs"] Oct 29 23:33:59.109040 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:59.108957 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:33:59.198866 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:59.198831 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 29 23:33:59.199028 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:33:59.198887 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" podUID="6d812120-3b8e-46c4-88d7-febeea4f0670" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 29 23:34:00.199198 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:00.199160 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 29 23:34:00.199618 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:00.199212 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" podUID="6d812120-3b8e-46c4-88d7-febeea4f0670" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 29 23:34:00.772029 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:00.771995 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3c8f079-e7ec-4d31-9477-e9edc61446db" path="/var/lib/kubelet/pods/d3c8f079-e7ec-4d31-9477-e9edc61446db/volumes" Oct 29 23:34:00.963845 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:00.963804 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:01.199057 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:01.199022 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 29 23:34:01.199228 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:01.199076 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" podUID="6d812120-3b8e-46c4-88d7-febeea4f0670" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 29 23:34:02.198891 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:02.198856 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 29 23:34:02.199096 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:02.198911 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" podUID="6d812120-3b8e-46c4-88d7-febeea4f0670" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 29 23:34:02.855343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:02.854497 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:03.083515 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:03.083478 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:03.459963 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:03.459926 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:34:03.461636 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:03.461610 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:34:03.462035 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:03.462013 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:34:04.621970 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:04.621933 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:05.527783 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:05.527746 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:05.692643 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:05.692615 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:05.852325 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:05.849303 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9"] Oct 29 23:34:05.852325 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:05.849643 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" podUID="35b2936b-6eff-4905-97f3-fb3ef5bdb95c" containerName="istio-proxy" containerID="cri-o://a6766f97617b97ab18620a9c03b3cd99a31e9c237aef125321f6a288578d26c3" gracePeriod=30 Oct 29 23:34:05.854048 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:05.854020 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62"] Oct 29 23:34:05.858308 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:05.854515 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" podUID="b9e7834a-0585-49fc-ba76-b68a2a1a48d3" containerName="istio-proxy" containerID="cri-o://fef4f4e901393e0a9e8e4267edaa859da34c236ae50f3cae51b99a3de2e2dde5" gracePeriod=30 Oct 29 23:34:05.860826 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:05.859731 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw"] Oct 29 23:34:05.860826 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:05.860008 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" podUID="ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" containerName="istio-proxy" containerID="cri-o://460c0a3d4e898311f617251e4c1726b27b6e10999053c96e8cd742d330d8cbc8" gracePeriod=30 Oct 29 23:34:05.865384 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:05.865362 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6"] Oct 29 23:34:05.865715 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:05.865691 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" podUID="b7d71a96-9322-4a8e-9384-6b104852a3b9" containerName="istio-proxy" containerID="cri-o://b6fadb3c2a89c5357276d82735dfc3cfd23d95be54e8c2179d31294eadb1a780" gracePeriod=30 Oct 29 23:34:05.876880 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:05.876856 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:06.587778 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:06.587728 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:09.161295 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:09.160580 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" podUID="da79ef4d-119b-4eed-870b-5e0fb48e60c8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 29 23:34:09.452790 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:09.452662 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:09.646286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:09.641691 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:10.544285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:10.530091 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:11.198318 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:11.153757 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" podUID="da79ef4d-119b-4eed-870b-5e0fb48e60c8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": context deadline exceeded" Oct 29 23:34:11.198318 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:11.177416 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:34:11.366288 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:11.341495 5056 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:34:12.193947 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:12.190700 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" podUID="da79ef4d-119b-4eed-870b-5e0fb48e60c8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:34:12.193947 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:12.192716 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:34:12.267725 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:12.228968 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:34:12.267725 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:12.252582 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:34:13.253551 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:13.249841 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": context deadline exceeded" Oct 29 23:34:13.377172 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:13.377134 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:34:13.377451 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:13.377421 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" podUID="1a24e9f0-98ce-4367-99b2-25653f0171ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:34:13.403804 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:13.403758 5056 generic.go:356] "Generic (PLEG): container finished" podID="b9e7834a-0585-49fc-ba76-b68a2a1a48d3" containerID="fef4f4e901393e0a9e8e4267edaa859da34c236ae50f3cae51b99a3de2e2dde5" exitCode=0 Oct 29 23:34:13.404738 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:13.404712 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:13.693641 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:13.693605 5056 generic.go:356] "Generic (PLEG): container finished" podID="35b2936b-6eff-4905-97f3-fb3ef5bdb95c" containerID="a6766f97617b97ab18620a9c03b3cd99a31e9c237aef125321f6a288578d26c3" exitCode=0 Oct 29 23:34:13.759822 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:13.759788 5056 generic.go:356] "Generic (PLEG): container finished" podID="ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" containerID="460c0a3d4e898311f617251e4c1726b27b6e10999053c96e8cd742d330d8cbc8" exitCode=0 Oct 29 23:34:13.762588 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:13.762559 5056 generic.go:356] "Generic (PLEG): container finished" podID="b7d71a96-9322-4a8e-9384-6b104852a3b9" containerID="b6fadb3c2a89c5357276d82735dfc3cfd23d95be54e8c2179d31294eadb1a780" exitCode=0 Oct 29 23:34:13.847340 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:13.844346 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:34:13.847340 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:13.844495 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" podUID="b46bbd7f-4b97-4b30-b500-1c31f0507898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:34:13.902310 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:34:13.877384 5056 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.078s" Oct 29 23:34:13.902581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:13.902546 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" event={"ID":"b9e7834a-0585-49fc-ba76-b68a2a1a48d3","Type":"ContainerDied","Data":"fef4f4e901393e0a9e8e4267edaa859da34c236ae50f3cae51b99a3de2e2dde5"} Oct 29 23:34:13.902790 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:13.902772 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:34:13.902903 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:13.902887 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" event={"ID":"35b2936b-6eff-4905-97f3-fb3ef5bdb95c","Type":"ContainerDied","Data":"a6766f97617b97ab18620a9c03b3cd99a31e9c237aef125321f6a288578d26c3"} Oct 29 23:34:13.903024 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:13.903004 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" event={"ID":"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5","Type":"ContainerDied","Data":"460c0a3d4e898311f617251e4c1726b27b6e10999053c96e8cd742d330d8cbc8"} Oct 29 23:34:13.903136 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:13.903122 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" event={"ID":"b7d71a96-9322-4a8e-9384-6b104852a3b9","Type":"ContainerDied","Data":"b6fadb3c2a89c5357276d82735dfc3cfd23d95be54e8c2179d31294eadb1a780"} Oct 29 23:34:14.039286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.018458 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" podUID="da79ef4d-119b-4eed-870b-5e0fb48e60c8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": context deadline exceeded" Oct 29 23:34:14.319749 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.319719 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:34:14.680285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.677591 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:34:14.750021 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.749986 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b7d71a96-9322-4a8e-9384-6b104852a3b9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7d71a96-9322-4a8e-9384-6b104852a3b9-istiod-ca-cert\") pod \"b7d71a96-9322-4a8e-9384-6b104852a3b9\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " Oct 29 23:34:14.759800 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.759766 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-workload-certs\") pod \"b7d71a96-9322-4a8e-9384-6b104852a3b9\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " Oct 29 23:34:14.760398 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.760377 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-credential-socket\") pod \"b7d71a96-9322-4a8e-9384-6b104852a3b9\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " Oct 29 23:34:14.760837 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.760822 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-token\" (UniqueName: \"kubernetes.io/projected/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-token\") pod \"b7d71a96-9322-4a8e-9384-6b104852a3b9\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " Oct 29 23:34:14.772694 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.772666 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-workload-socket\") pod \"b7d71a96-9322-4a8e-9384-6b104852a3b9\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " Oct 29 23:34:14.783482 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.783458 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-podinfo\") pod \"b7d71a96-9322-4a8e-9384-6b104852a3b9\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " Oct 29 23:34:14.783994 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.783975 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-data\") pod \"b7d71a96-9322-4a8e-9384-6b104852a3b9\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " Oct 29 23:34:14.784466 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.784441 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7d71a96-9322-4a8e-9384-6b104852a3b9-kube-api-access-nfrjk\" (UniqueName: \"kubernetes.io/projected/b7d71a96-9322-4a8e-9384-6b104852a3b9-kube-api-access-nfrjk\") pod \"b7d71a96-9322-4a8e-9384-6b104852a3b9\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " Oct 29 23:34:14.784930 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.784913 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-envoy\") pod \"b7d71a96-9322-4a8e-9384-6b104852a3b9\" (UID: \"b7d71a96-9322-4a8e-9384-6b104852a3b9\") " Oct 29 23:34:14.785511 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.750585 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7d71a96-9322-4a8e-9384-6b104852a3b9-istiod-ca-cert" pod "b7d71a96-9322-4a8e-9384-6b104852a3b9" (UID: "b7d71a96-9322-4a8e-9384-6b104852a3b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:34:14.788397 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.760714 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-credential-socket" pod "b7d71a96-9322-4a8e-9384-6b104852a3b9" (UID: "b7d71a96-9322-4a8e-9384-6b104852a3b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:14.788551 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.760759 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-workload-certs" pod "b7d71a96-9322-4a8e-9384-6b104852a3b9" (UID: "b7d71a96-9322-4a8e-9384-6b104852a3b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:14.788691 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.781209 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-token" pod "b7d71a96-9322-4a8e-9384-6b104852a3b9" (UID: "b7d71a96-9322-4a8e-9384-6b104852a3b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:34:14.788818 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.781547 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-workload-socket" pod "b7d71a96-9322-4a8e-9384-6b104852a3b9" (UID: "b7d71a96-9322-4a8e-9384-6b104852a3b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:14.789107 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.784342 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-data" pod "b7d71a96-9322-4a8e-9384-6b104852a3b9" (UID: "b7d71a96-9322-4a8e-9384-6b104852a3b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:14.868741 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.849484 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-podinfo" pod "b7d71a96-9322-4a8e-9384-6b104852a3b9" (UID: "b7d71a96-9322-4a8e-9384-6b104852a3b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:34:14.868741 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.849706 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7d71a96-9322-4a8e-9384-6b104852a3b9-kube-api-access-nfrjk" pod "b7d71a96-9322-4a8e-9384-6b104852a3b9" (UID: "b7d71a96-9322-4a8e-9384-6b104852a3b9"). InnerVolumeSpecName "kube-api-access-nfrjk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:34:14.871058 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.870716 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-envoy" pod "b7d71a96-9322-4a8e-9384-6b104852a3b9" (UID: "b7d71a96-9322-4a8e-9384-6b104852a3b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:14.900671 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.900639 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7d71a96-9322-4a8e-9384-6b104852a3b9-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:14.906563 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.900882 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:14.906794 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.906773 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:14.906926 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.906912 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:14.907028 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.907017 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:14.907137 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.907125 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:14.907242 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.907231 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:14.907348 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.907335 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfrjk\" (UniqueName: \"kubernetes.io/projected/b7d71a96-9322-4a8e-9384-6b104852a3b9-kube-api-access-nfrjk\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:14.907457 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:14.907444 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7d71a96-9322-4a8e-9384-6b104852a3b9-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.003214 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.003104 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:34:15.150386 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.146562 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" Oct 29 23:34:15.150386 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.147293 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6" event={"ID":"b7d71a96-9322-4a8e-9384-6b104852a3b9","Type":"ContainerDied","Data":"554bbc393e52c07add2f7be0fbec7c132fef8374a3ed2100c8b6636969d6d988"} Oct 29 23:34:15.150386 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.147392 5056 scope.go:122] "RemoveContainer" containerID="b6fadb3c2a89c5357276d82735dfc3cfd23d95be54e8c2179d31294eadb1a780" Oct 29 23:34:15.173901 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.165388 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-podinfo\") pod \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " Oct 29 23:34:15.173901 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.165435 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-data\") pod \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " Oct 29 23:34:15.173901 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.165466 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-kube-api-access-6hz7p\" (UniqueName: \"kubernetes.io/projected/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-kube-api-access-6hz7p\") pod \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " Oct 29 23:34:15.173901 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.165516 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-workload-certs\") pod \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " Oct 29 23:34:15.173901 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.165600 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-envoy\") pod \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " Oct 29 23:34:15.173901 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.165627 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-workload-socket\") pod \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " Oct 29 23:34:15.173901 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.165681 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-token\" (UniqueName: \"kubernetes.io/projected/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-token\") pod \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " Oct 29 23:34:15.173901 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.165753 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istiod-ca-cert\") pod \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " Oct 29 23:34:15.173901 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.165802 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-credential-socket\") pod \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\" (UID: \"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5\") " Oct 29 23:34:15.173901 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.168952 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-kube-api-access-6hz7p" pod "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" (UID: "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5"). InnerVolumeSpecName "kube-api-access-6hz7p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:34:15.173901 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.169535 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-data" pod "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" (UID: "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:15.173901 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.171008 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-workload-socket" pod "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" (UID: "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:15.173901 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.171209 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-workload-certs" pod "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" (UID: "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:15.173901 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.173699 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-envoy" pod "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" (UID: "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:15.174800 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.174334 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-podinfo" pod "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" (UID: "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:34:15.174800 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.174655 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istiod-ca-cert" pod "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" (UID: "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:34:15.175350 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.175327 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-credential-socket" pod "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" (UID: "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:15.177741 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.177713 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.177865 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.177743 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.177865 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.177758 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hz7p\" (UniqueName: \"kubernetes.io/projected/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-kube-api-access-6hz7p\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.177865 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.177772 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.177865 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.177784 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.177865 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.177796 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.177865 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.177807 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.177865 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.177820 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.178719 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.178639 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-token" pod "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" (UID: "ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:34:15.182136 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.182119 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:34:15.184426 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.184395 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:34:15.210470 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.210427 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6"] Oct 29 23:34:15.214901 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.214875 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-w4ss6"] Oct 29 23:34:15.278712 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.278681 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-workload-certs\") pod \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " Oct 29 23:34:15.278961 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.278938 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-credential-socket\") pod \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " Oct 29 23:34:15.279394 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.279096 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-workload-certs" pod "b9e7834a-0585-49fc-ba76-b68a2a1a48d3" (UID: "b9e7834a-0585-49fc-ba76-b68a2a1a48d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:15.279489 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.279302 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-credential-socket" pod "35b2936b-6eff-4905-97f3-fb3ef5bdb95c" (UID: "35b2936b-6eff-4905-97f3-fb3ef5bdb95c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:15.279574 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.279373 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-podinfo\") pod \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " Oct 29 23:34:15.279704 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.279610 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-credential-socket\") pod \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " Oct 29 23:34:15.279704 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.279644 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-token\" (UniqueName: \"kubernetes.io/projected/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-token\") pod \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " Oct 29 23:34:15.279704 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.279673 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-podinfo\") pod \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " Oct 29 23:34:15.279872 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.279706 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istiod-ca-cert\") pod \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " Oct 29 23:34:15.279872 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.279755 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-token\" (UniqueName: \"kubernetes.io/projected/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-token\") pod \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " Oct 29 23:34:15.279872 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.279795 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-kube-api-access-dkrn6\" (UniqueName: \"kubernetes.io/projected/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-kube-api-access-dkrn6\") pod \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " Oct 29 23:34:15.279872 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.279851 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-workload-certs\") pod \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " Oct 29 23:34:15.280080 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.279927 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-envoy\") pod \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " Oct 29 23:34:15.280080 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.279979 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-workload-socket\") pod \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " Oct 29 23:34:15.280080 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.280074 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-data\") pod \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " Oct 29 23:34:15.280226 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.280108 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istiod-ca-cert\") pod \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " Oct 29 23:34:15.280226 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.280136 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-workload-socket\") pod \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " Oct 29 23:34:15.280226 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.280202 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-envoy\") pod \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " Oct 29 23:34:15.280399 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.280251 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-data\") pod \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\" (UID: \"35b2936b-6eff-4905-97f3-fb3ef5bdb95c\") " Oct 29 23:34:15.280399 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.280339 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-kube-api-access-pkjt4\" (UniqueName: \"kubernetes.io/projected/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-kube-api-access-pkjt4\") pod \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\" (UID: \"b9e7834a-0585-49fc-ba76-b68a2a1a48d3\") " Oct 29 23:34:15.281063 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.281041 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.281140 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.281070 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.281140 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.281085 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.281566 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.281505 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-credential-socket" pod "b9e7834a-0585-49fc-ba76-b68a2a1a48d3" (UID: "b9e7834a-0585-49fc-ba76-b68a2a1a48d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:15.281650 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.281603 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-data" pod "b9e7834a-0585-49fc-ba76-b68a2a1a48d3" (UID: "b9e7834a-0585-49fc-ba76-b68a2a1a48d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:15.281709 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.281686 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istiod-ca-cert" pod "35b2936b-6eff-4905-97f3-fb3ef5bdb95c" (UID: "35b2936b-6eff-4905-97f3-fb3ef5bdb95c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:34:15.281814 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.281794 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-workload-socket" pod "b9e7834a-0585-49fc-ba76-b68a2a1a48d3" (UID: "b9e7834a-0585-49fc-ba76-b68a2a1a48d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:15.282033 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.282009 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-workload-certs" pod "35b2936b-6eff-4905-97f3-fb3ef5bdb95c" (UID: "35b2936b-6eff-4905-97f3-fb3ef5bdb95c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:15.282249 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.282227 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istiod-ca-cert" pod "b9e7834a-0585-49fc-ba76-b68a2a1a48d3" (UID: "b9e7834a-0585-49fc-ba76-b68a2a1a48d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:34:15.282480 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.282383 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-data" pod "35b2936b-6eff-4905-97f3-fb3ef5bdb95c" (UID: "35b2936b-6eff-4905-97f3-fb3ef5bdb95c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:15.282566 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.282504 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-workload-socket" pod "35b2936b-6eff-4905-97f3-fb3ef5bdb95c" (UID: "35b2936b-6eff-4905-97f3-fb3ef5bdb95c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:15.284538 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.284511 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-token" pod "35b2936b-6eff-4905-97f3-fb3ef5bdb95c" (UID: "35b2936b-6eff-4905-97f3-fb3ef5bdb95c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:34:15.288011 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.287976 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-envoy" pod "b9e7834a-0585-49fc-ba76-b68a2a1a48d3" (UID: "b9e7834a-0585-49fc-ba76-b68a2a1a48d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:15.288011 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.287987 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-envoy" pod "35b2936b-6eff-4905-97f3-fb3ef5bdb95c" (UID: "35b2936b-6eff-4905-97f3-fb3ef5bdb95c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:15.288159 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.288040 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-kube-api-access-pkjt4" pod "b9e7834a-0585-49fc-ba76-b68a2a1a48d3" (UID: "b9e7834a-0585-49fc-ba76-b68a2a1a48d3"). InnerVolumeSpecName "kube-api-access-pkjt4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:34:15.288159 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.288096 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-podinfo" pod "35b2936b-6eff-4905-97f3-fb3ef5bdb95c" (UID: "35b2936b-6eff-4905-97f3-fb3ef5bdb95c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:34:15.288284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.288247 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-podinfo" pod "b9e7834a-0585-49fc-ba76-b68a2a1a48d3" (UID: "b9e7834a-0585-49fc-ba76-b68a2a1a48d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:34:15.289174 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.289150 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-token" pod "b9e7834a-0585-49fc-ba76-b68a2a1a48d3" (UID: "b9e7834a-0585-49fc-ba76-b68a2a1a48d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:34:15.289673 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.289649 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-kube-api-access-dkrn6" pod "35b2936b-6eff-4905-97f3-fb3ef5bdb95c" (UID: "35b2936b-6eff-4905-97f3-fb3ef5bdb95c"). InnerVolumeSpecName "kube-api-access-dkrn6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:34:15.382497 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.382466 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.382497 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.382494 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.382497 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.382504 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.382497 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.382513 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.382960 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.382521 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.382960 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.382529 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.382960 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.382538 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkrn6\" (UniqueName: \"kubernetes.io/projected/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-kube-api-access-dkrn6\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.382960 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.382547 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.382960 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.382555 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.382960 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.382562 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.382960 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.382570 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.382960 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.382578 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.382960 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.382585 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.382960 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.382592 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.382960 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.382600 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/35b2936b-6eff-4905-97f3-fb3ef5bdb95c-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:15.382960 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:15.382607 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkjt4\" (UniqueName: \"kubernetes.io/projected/b9e7834a-0585-49fc-ba76-b68a2a1a48d3-kube-api-access-pkjt4\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:16.154170 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.154141 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" Oct 29 23:34:16.154387 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.154173 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62" event={"ID":"b9e7834a-0585-49fc-ba76-b68a2a1a48d3","Type":"ContainerDied","Data":"9e54060db461a220a0032a0dc5db10d192600cdee4218ef5e98ccff9f797c0b5"} Oct 29 23:34:16.154387 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.154249 5056 scope.go:122] "RemoveContainer" containerID="fef4f4e901393e0a9e8e4267edaa859da34c236ae50f3cae51b99a3de2e2dde5" Oct 29 23:34:16.155824 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.155806 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" Oct 29 23:34:16.155824 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.155810 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9" event={"ID":"35b2936b-6eff-4905-97f3-fb3ef5bdb95c","Type":"ContainerDied","Data":"e4d685fb42ce925aa9e6683d8b83c4f71b42460b5dda8dc06448c1e5c0c244cd"} Oct 29 23:34:16.157631 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.157612 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" Oct 29 23:34:16.157757 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.157652 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw" event={"ID":"ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5","Type":"ContainerDied","Data":"b7b02fd8282e44eff4cdf07e1febd2b583ae8b71ccd5cfd10ffa034e73796183"} Oct 29 23:34:16.166973 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.166956 5056 scope.go:122] "RemoveContainer" containerID="a6766f97617b97ab18620a9c03b3cd99a31e9c237aef125321f6a288578d26c3" Oct 29 23:34:16.176451 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.176425 5056 scope.go:122] "RemoveContainer" containerID="460c0a3d4e898311f617251e4c1726b27b6e10999053c96e8cd742d330d8cbc8" Oct 29 23:34:16.223467 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.223438 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62"] Oct 29 23:34:16.250104 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.250054 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-jbv62"] Oct 29 23:34:16.286304 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.282781 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:16.301316 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.301277 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9"] Oct 29 23:34:16.311394 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.311348 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-kdhl9"] Oct 29 23:34:16.334927 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.334895 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw"] Oct 29 23:34:16.340572 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.340542 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-b978a0-openshift-default-5d657d9dff-vtfrw"] Oct 29 23:34:16.773474 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.773441 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35b2936b-6eff-4905-97f3-fb3ef5bdb95c" path="/var/lib/kubelet/pods/35b2936b-6eff-4905-97f3-fb3ef5bdb95c/volumes" Oct 29 23:34:16.773962 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.773948 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b7d71a96-9322-4a8e-9384-6b104852a3b9" path="/var/lib/kubelet/pods/b7d71a96-9322-4a8e-9384-6b104852a3b9/volumes" Oct 29 23:34:16.774393 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.774379 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9e7834a-0585-49fc-ba76-b68a2a1a48d3" path="/var/lib/kubelet/pods/b9e7834a-0585-49fc-ba76-b68a2a1a48d3/volumes" Oct 29 23:34:16.775132 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:16.775117 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" path="/var/lib/kubelet/pods/ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5/volumes" Oct 29 23:34:18.045957 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:18.038211 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:18.064672 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:18.064637 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n"] Oct 29 23:34:18.215336 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:18.215300 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:19.904283 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:19.903361 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:21.759323 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:21.758185 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:22.416554 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:34:22.416514 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:34:23.552920 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:23.552867 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:24.217238 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.217204 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk"] Oct 29 23:34:24.378772 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.378730 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:24.386186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.386159 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h"] Oct 29 23:34:24.386792 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.386776 5056 cpu_manager.go:414] "containerMap: removing container" podUID="35b2936b-6eff-4905-97f3-fb3ef5bdb95c" containerName="istio-proxy" Oct 29 23:34:24.386792 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.386793 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35b2936b-6eff-4905-97f3-fb3ef5bdb95c" containerName="istio-proxy" Oct 29 23:34:24.386891 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.386808 5056 cpu_manager.go:414] "containerMap: removing container" podUID="ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" containerName="istio-proxy" Oct 29 23:34:24.386891 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.386814 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" containerName="istio-proxy" Oct 29 23:34:24.386891 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.386832 5056 cpu_manager.go:414] "containerMap: removing container" podUID="d3c8f079-e7ec-4d31-9477-e9edc61446db" containerName="maas-api" Oct 29 23:34:24.386891 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.386837 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3c8f079-e7ec-4d31-9477-e9edc61446db" containerName="maas-api" Oct 29 23:34:24.386891 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.386845 5056 cpu_manager.go:414] "containerMap: removing container" podUID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerName="istio-proxy" Oct 29 23:34:24.386891 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.386850 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerName="istio-proxy" Oct 29 23:34:24.386891 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.386858 5056 cpu_manager.go:414] "containerMap: removing container" podUID="b9e7834a-0585-49fc-ba76-b68a2a1a48d3" containerName="istio-proxy" Oct 29 23:34:24.386891 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.386863 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9e7834a-0585-49fc-ba76-b68a2a1a48d3" containerName="istio-proxy" Oct 29 23:34:24.386891 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.386879 5056 cpu_manager.go:414] "containerMap: removing container" podUID="b7d71a96-9322-4a8e-9384-6b104852a3b9" containerName="istio-proxy" Oct 29 23:34:24.386891 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.386884 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7d71a96-9322-4a8e-9384-6b104852a3b9" containerName="istio-proxy" Oct 29 23:34:24.387177 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.386974 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="35b2936b-6eff-4905-97f3-fb3ef5bdb95c" containerName="istio-proxy" Oct 29 23:34:24.387177 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.386981 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="ecc9a4a6-6e93-4054-9bf2-cb1509bf2eb5" containerName="istio-proxy" Oct 29 23:34:24.387177 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.386997 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3c8f079-e7ec-4d31-9477-e9edc61446db" containerName="maas-api" Oct 29 23:34:24.387177 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.387006 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9e7834a-0585-49fc-ba76-b68a2a1a48d3" containerName="istio-proxy" Oct 29 23:34:24.387177 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.387018 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7d71a96-9322-4a8e-9384-6b104852a3b9" containerName="istio-proxy" Oct 29 23:34:24.387177 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.387027 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="40c7dc3a-342c-48a9-a569-d9545b1a6653" containerName="istio-proxy" Oct 29 23:34:24.392078 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.392061 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.452558 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.452513 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.452773 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.452573 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd4bz\" (UniqueName: \"kubernetes.io/projected/9369cf81-a466-4661-b370-e676e87abd4b-kube-api-access-fd4bz\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.452773 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.452619 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9369cf81-a466-4661-b370-e676e87abd4b-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.452773 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.452680 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.452773 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.452732 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.453047 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.452781 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.453047 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.452805 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.453047 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.452939 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9369cf81-a466-4661-b370-e676e87abd4b-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.453047 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.453014 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9369cf81-a466-4661-b370-e676e87abd4b-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.493680 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.493504 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj"] Oct 29 23:34:24.546335 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.525467 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.550385 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.550356 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h"] Oct 29 23:34:24.554658 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.554626 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.555274 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.555237 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.557297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.555388 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj"] Oct 29 23:34:24.557506 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.555771 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.557506 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.557140 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:24.561327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.557699 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.561327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.557734 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.561327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.557862 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9369cf81-a466-4661-b370-e676e87abd4b-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.561327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.557935 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9369cf81-a466-4661-b370-e676e87abd4b-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.561327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.558031 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.561327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.558063 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd4bz\" (UniqueName: \"kubernetes.io/projected/9369cf81-a466-4661-b370-e676e87abd4b-kube-api-access-fd4bz\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.561327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.558109 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9369cf81-a466-4661-b370-e676e87abd4b-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.561327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.559572 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.561327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.559838 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.561327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.560047 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.561327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.560237 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.562604 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.562581 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9369cf81-a466-4661-b370-e676e87abd4b-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.564834 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.564806 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9369cf81-a466-4661-b370-e676e87abd4b-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.604123 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.604084 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9369cf81-a466-4661-b370-e676e87abd4b-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.604298 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.604206 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd4bz\" (UniqueName: \"kubernetes.io/projected/9369cf81-a466-4661-b370-e676e87abd4b-kube-api-access-fd4bz\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.659399 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.659363 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8622875-ba05-4797-92ce-4e208a3a0540-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.659399 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.659398 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.659611 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.659531 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.659685 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.659663 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8622875-ba05-4797-92ce-4e208a3a0540-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.659736 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.659727 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.659775 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.659747 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8622875-ba05-4797-92ce-4e208a3a0540-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.659862 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.659846 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gldx\" (UniqueName: \"kubernetes.io/projected/d8622875-ba05-4797-92ce-4e208a3a0540-kube-api-access-4gldx\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.659944 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.659915 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.659991 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.659971 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.712691 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.712667 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:24.760899 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.760824 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gldx\" (UniqueName: \"kubernetes.io/projected/d8622875-ba05-4797-92ce-4e208a3a0540-kube-api-access-4gldx\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.760899 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.760877 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.760899 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.760896 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.761155 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.760955 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8622875-ba05-4797-92ce-4e208a3a0540-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.761155 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.760976 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.761155 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.761057 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.761342 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.761158 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8622875-ba05-4797-92ce-4e208a3a0540-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.761342 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.761214 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.761342 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.761252 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8622875-ba05-4797-92ce-4e208a3a0540-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.761342 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.761290 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-credential-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.761463 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.761367 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-istio-data\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.761643 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.761618 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-workload-certs\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.761712 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.761679 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-workload-socket\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.761919 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.761898 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8622875-ba05-4797-92ce-4e208a3a0540-istiod-ca-cert\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.763733 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.763709 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-istio-envoy\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.765616 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.765591 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8622875-ba05-4797-92ce-4e208a3a0540-istio-podinfo\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.794700 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.794668 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8622875-ba05-4797-92ce-4e208a3a0540-istio-token\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.797953 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.797931 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gldx\" (UniqueName: \"kubernetes.io/projected/d8622875-ba05-4797-92ce-4e208a3a0540-kube-api-access-4gldx\") pod \"e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:24.915284 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:24.909571 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:25.110961 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:34:25.110910 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9369cf81_a466_4661_b370_e676e87abd4b.slice/crio-7b7f201b74b077822d90ba1951d02a5b34dcf4b87e736919b381cdb7c5d7cbdd WatchSource:0}: Error finding container 7b7f201b74b077822d90ba1951d02a5b34dcf4b87e736919b381cdb7c5d7cbdd: Status 404 returned error can't find the container with id 7b7f201b74b077822d90ba1951d02a5b34dcf4b87e736919b381cdb7c5d7cbdd Oct 29 23:34:25.113636 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:25.113593 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h"] Oct 29 23:34:25.114750 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:25.114707 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:34:25.114857 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:25.114789 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:34:25.114857 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:25.114839 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:34:25.239319 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:25.239276 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" event={"ID":"9369cf81-a466-4661-b370-e676e87abd4b","Type":"ContainerStarted","Data":"ef78b9dd06e4184263dfab3dfde4b1a32bd84adbb83c91009cc1365b3d986e2b"} Oct 29 23:34:25.239501 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:25.239348 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" event={"ID":"9369cf81-a466-4661-b370-e676e87abd4b","Type":"ContainerStarted","Data":"7b7f201b74b077822d90ba1951d02a5b34dcf4b87e736919b381cdb7c5d7cbdd"} Oct 29 23:34:25.292430 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:25.292371 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" podStartSLOduration=1.292356476 podStartE2EDuration="1.292356476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:34:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:34:25.287748784 +0000 UTC m=+1292.654496418" watchObservedRunningTime="2026-10-29 23:34:25.292356476 +0000 UTC m=+1292.659104182" Oct 29 23:34:25.317478 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:25.317452 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj"] Oct 29 23:34:25.335702 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:25.335479 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:34:25.335702 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:25.335596 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:34:25.335702 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:25.335681 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:34:25.713693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:25.713589 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:25.753119 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:25.753069 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:26.282174 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:26.282132 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" event={"ID":"d8622875-ba05-4797-92ce-4e208a3a0540","Type":"ContainerStarted","Data":"8a1d5b171e3633570c48bdf16d25895e85583c9af83fc18bc32021075ed07b9a"} Oct 29 23:34:26.282396 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:26.282245 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" event={"ID":"d8622875-ba05-4797-92ce-4e208a3a0540","Type":"ContainerStarted","Data":"4a45efbf7f2fdc5b7567cd132bb2218815ac861f04619083abdf733cf21473c1"} Oct 29 23:34:26.727433 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:26.727396 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body= Oct 29 23:34:26.728057 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:26.728018 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" podUID="9369cf81-a466-4661-b370-e676e87abd4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" Oct 29 23:34:26.910241 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:26.910205 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:26.911771 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:26.911748 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 23:34:26.911843 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:26.911793 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" podUID="d8622875-ba05-4797-92ce-4e208a3a0540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 23:34:27.714094 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:27.714055 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body= Oct 29 23:34:27.714420 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:27.714385 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" podUID="9369cf81-a466-4661-b370-e676e87abd4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" Oct 29 23:34:27.734801 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:27.734461 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:27.910091 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:27.910048 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 23:34:27.910240 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:27.910110 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" podUID="d8622875-ba05-4797-92ce-4e208a3a0540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 23:34:28.713060 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:28.713027 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body= Oct 29 23:34:28.713300 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:28.713076 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" podUID="9369cf81-a466-4661-b370-e676e87abd4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" Oct 29 23:34:28.911642 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:28.911613 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 23:34:28.912100 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:28.912067 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" podUID="d8622875-ba05-4797-92ce-4e208a3a0540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 23:34:29.230362 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:29.225327 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" podUID="4cb6d795-205b-465f-91b0-fa184fa8fe44" containerName="payload-pre-processing" containerID="cri-o://f9e75e3916a6ba4fcd6426fd98138674c80f81b5bdf4aad07ac2e244de5d0154" gracePeriod=25 Oct 29 23:34:29.720287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:29.718677 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body= Oct 29 23:34:29.720287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:29.718738 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" podUID="9369cf81-a466-4661-b370-e676e87abd4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" Oct 29 23:34:29.910437 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:29.910392 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 23:34:29.910593 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:29.910449 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" podUID="d8622875-ba05-4797-92ce-4e208a3a0540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 23:34:30.006469 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:30.005248 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:30.713158 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:30.713124 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body= Oct 29 23:34:30.713396 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:30.713172 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" podUID="9369cf81-a466-4661-b370-e676e87abd4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" Oct 29 23:34:30.910551 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:30.910517 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 23:34:30.910707 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:30.910578 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" podUID="d8622875-ba05-4797-92ce-4e208a3a0540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 23:34:31.741568 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:31.741530 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" start-of-body= Oct 29 23:34:31.742150 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:31.742118 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" podUID="9369cf81-a466-4661-b370-e676e87abd4b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.92:15021/healthz/ready\": dial tcp 10.132.0.92:15021: connect: connection refused" Oct 29 23:34:31.910873 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:31.910832 5056 patch_prober.go:29] interesting pod/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body= Oct 29 23:34:31.911034 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:31.910904 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" podUID="d8622875-ba05-4797-92ce-4e208a3a0540" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" Oct 29 23:34:32.083335 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:32.069519 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:32.803508 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:32.803472 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:32.804571 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:32.804531 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:32.809430 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:32.809408 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:34:32.892660 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:32.892598 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" podStartSLOduration=8.892580253 podStartE2EDuration="8.892580253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:34:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:34:26.362479024 +0000 UTC m=+1293.729226656" watchObservedRunningTime="2026-10-29 23:34:32.892580253 +0000 UTC m=+1300.259327885" Oct 29 23:34:33.131772 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.131670 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" containerID="cri-o://5cf955376e9f8d7dce5a972989bf9440d75c5a0ee75cc6f64781cd5eddd70a7e" gracePeriod=45 Oct 29 23:34:33.196572 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.196189 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:33.197435 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.197409 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:33.198405 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.198386 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:34:33.340087 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.340053 5056 generic.go:356] "Generic (PLEG): container finished" podID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerID="5cf955376e9f8d7dce5a972989bf9440d75c5a0ee75cc6f64781cd5eddd70a7e" exitCode=0 Oct 29 23:34:33.341371 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.341342 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" event={"ID":"a267a091-f7f9-4f00-bf02-5fd668eb27c4","Type":"ContainerDied","Data":"5cf955376e9f8d7dce5a972989bf9440d75c5a0ee75cc6f64781cd5eddd70a7e"} Oct 29 23:34:33.454994 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.454963 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:34:33.511590 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.511533 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a267a091-f7f9-4f00-bf02-5fd668eb27c4-tls-certs\" (UniqueName: \"kubernetes.io/secret/a267a091-f7f9-4f00-bf02-5fd668eb27c4-tls-certs\") pod \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " Oct 29 23:34:33.511729 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.511607 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-home\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-home\") pod \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " Oct 29 23:34:33.511729 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.511645 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-tmp-dir\") pod \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " Oct 29 23:34:33.511729 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.511715 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a267a091-f7f9-4f00-bf02-5fd668eb27c4-kube-api-access-fmxsj\" (UniqueName: \"kubernetes.io/projected/a267a091-f7f9-4f00-bf02-5fd668eb27c4-kube-api-access-fmxsj\") pod \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " Oct 29 23:34:33.512017 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.511996 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-model-cache\") pod \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " Oct 29 23:34:33.512122 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.512030 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-home" pod "a267a091-f7f9-4f00-bf02-5fd668eb27c4" (UID: "a267a091-f7f9-4f00-bf02-5fd668eb27c4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:33.512122 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.512061 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-dshm\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-dshm\") pod \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\" (UID: \"a267a091-f7f9-4f00-bf02-5fd668eb27c4\") " Oct 29 23:34:33.512650 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.512627 5056 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-home\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:33.513248 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.513144 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-model-cache" pod "a267a091-f7f9-4f00-bf02-5fd668eb27c4" (UID: "a267a091-f7f9-4f00-bf02-5fd668eb27c4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:33.513425 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.513202 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-tmp-dir" pod "a267a091-f7f9-4f00-bf02-5fd668eb27c4" (UID: "a267a091-f7f9-4f00-bf02-5fd668eb27c4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:33.516703 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.516667 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a267a091-f7f9-4f00-bf02-5fd668eb27c4-tls-certs" pod "a267a091-f7f9-4f00-bf02-5fd668eb27c4" (UID: "a267a091-f7f9-4f00-bf02-5fd668eb27c4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:34:33.517095 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.517071 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a267a091-f7f9-4f00-bf02-5fd668eb27c4-kube-api-access-fmxsj" pod "a267a091-f7f9-4f00-bf02-5fd668eb27c4" (UID: "a267a091-f7f9-4f00-bf02-5fd668eb27c4"). InnerVolumeSpecName "kube-api-access-fmxsj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:34:33.520768 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.520748 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-dshm" pod "a267a091-f7f9-4f00-bf02-5fd668eb27c4" (UID: "a267a091-f7f9-4f00-bf02-5fd668eb27c4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:34:33.624974 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.620585 5056 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-model-cache\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:33.624974 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.620623 5056 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-dshm\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:33.624974 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.620635 5056 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a267a091-f7f9-4f00-bf02-5fd668eb27c4-tls-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:33.624974 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.620645 5056 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a267a091-f7f9-4f00-bf02-5fd668eb27c4-tmp-dir\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:33.624974 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:33.620658 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmxsj\" (UniqueName: \"kubernetes.io/projected/a267a091-f7f9-4f00-bf02-5fd668eb27c4-kube-api-access-fmxsj\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:34.162344 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:34.159751 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:34.347449 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:34.347405 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.81:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 29 23:34:34.348853 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:34.348826 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" Oct 29 23:34:34.348956 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:34.348871 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n" event={"ID":"a267a091-f7f9-4f00-bf02-5fd668eb27c4","Type":"ContainerDied","Data":"c3116dbc791fa83cdb6d25b6ffbf1367c31e14b5839232284bee465306e14947"} Oct 29 23:34:34.348956 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:34.348921 5056 scope.go:122] "RemoveContainer" containerID="5cf955376e9f8d7dce5a972989bf9440d75c5a0ee75cc6f64781cd5eddd70a7e" Oct 29 23:34:34.403379 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:34.403353 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n"] Oct 29 23:34:34.407725 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:34.407690 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-2a53c9/distinct-w1-2a53c9-kserve-78c8d9d945-n529n"] Oct 29 23:34:34.770713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:34.770680 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" path="/var/lib/kubelet/pods/a267a091-f7f9-4f00-bf02-5fd668eb27c4/volumes" Oct 29 23:34:35.936302 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:35.935873 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:36.940038 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:36.939997 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" podUID="4cb6d795-205b-465f-91b0-fa184fa8fe44" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 29 23:34:38.555803 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:38.555768 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:40.484298 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:40.483657 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:42.268294 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:42.268238 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:44.123170 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:44.123133 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:44.351183 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:44.349935 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:45.651805 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:45.651771 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:46.032537 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.032449 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7"] Oct 29 23:34:46.032750 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.032707 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" podUID="c103618d-ee4d-4568-bdaf-6e1b486f9277" containerName="maas-api" containerID="cri-o://b5102643cb97930058c6eae715e9aa7640f2a3d241242e73e1421f438e834c79" gracePeriod=30 Oct 29 23:34:46.164331 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.164301 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:46.483285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.482706 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:34:46.795137 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.787435 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c103618d-ee4d-4568-bdaf-6e1b486f9277-kube-api-access-mv2dj\" (UniqueName: \"kubernetes.io/projected/c103618d-ee4d-4568-bdaf-6e1b486f9277-kube-api-access-mv2dj\") pod \"c103618d-ee4d-4568-bdaf-6e1b486f9277\" (UID: \"c103618d-ee4d-4568-bdaf-6e1b486f9277\") " Oct 29 23:34:46.795137 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.787530 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c103618d-ee4d-4568-bdaf-6e1b486f9277-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c103618d-ee4d-4568-bdaf-6e1b486f9277-metrics-certs\") pod \"c103618d-ee4d-4568-bdaf-6e1b486f9277\" (UID: \"c103618d-ee4d-4568-bdaf-6e1b486f9277\") " Oct 29 23:34:46.795137 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.787614 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c103618d-ee4d-4568-bdaf-6e1b486f9277-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c103618d-ee4d-4568-bdaf-6e1b486f9277-maas-api-tls\") pod \"c103618d-ee4d-4568-bdaf-6e1b486f9277\" (UID: \"c103618d-ee4d-4568-bdaf-6e1b486f9277\") " Oct 29 23:34:46.795137 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.787786 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c103618d-ee4d-4568-bdaf-6e1b486f9277-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c103618d-ee4d-4568-bdaf-6e1b486f9277-openshift-service-ca\") pod \"c103618d-ee4d-4568-bdaf-6e1b486f9277\" (UID: \"c103618d-ee4d-4568-bdaf-6e1b486f9277\") " Oct 29 23:34:46.795137 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.792857 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c103618d-ee4d-4568-bdaf-6e1b486f9277-kube-api-access-mv2dj" pod "c103618d-ee4d-4568-bdaf-6e1b486f9277" (UID: "c103618d-ee4d-4568-bdaf-6e1b486f9277"). InnerVolumeSpecName "kube-api-access-mv2dj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:34:46.822315 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.798115 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c103618d-ee4d-4568-bdaf-6e1b486f9277-metrics-certs" pod "c103618d-ee4d-4568-bdaf-6e1b486f9277" (UID: "c103618d-ee4d-4568-bdaf-6e1b486f9277"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:34:46.822315 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.814614 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c103618d-ee4d-4568-bdaf-6e1b486f9277-openshift-service-ca" pod "c103618d-ee4d-4568-bdaf-6e1b486f9277" (UID: "c103618d-ee4d-4568-bdaf-6e1b486f9277"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:34:46.822315 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.820514 5056 generic.go:356] "Generic (PLEG): container finished" podID="c103618d-ee4d-4568-bdaf-6e1b486f9277" containerID="b5102643cb97930058c6eae715e9aa7640f2a3d241242e73e1421f438e834c79" exitCode=0 Oct 29 23:34:46.822315 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.820694 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" Oct 29 23:34:46.827114 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.827073 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" event={"ID":"c103618d-ee4d-4568-bdaf-6e1b486f9277","Type":"ContainerDied","Data":"b5102643cb97930058c6eae715e9aa7640f2a3d241242e73e1421f438e834c79"} Oct 29 23:34:46.827301 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.827139 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7" event={"ID":"c103618d-ee4d-4568-bdaf-6e1b486f9277","Type":"ContainerDied","Data":"00b50f0dca3da3107e050b65341a98676b2c2b652ac09ee4145fc97050000aa4"} Oct 29 23:34:46.827301 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.827173 5056 scope.go:122] "RemoveContainer" containerID="b5102643cb97930058c6eae715e9aa7640f2a3d241242e73e1421f438e834c79" Oct 29 23:34:46.842528 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.842505 5056 scope.go:122] "RemoveContainer" containerID="b5102643cb97930058c6eae715e9aa7640f2a3d241242e73e1421f438e834c79" Oct 29 23:34:46.842879 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:34:46.842854 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5102643cb97930058c6eae715e9aa7640f2a3d241242e73e1421f438e834c79\": container with ID starting with b5102643cb97930058c6eae715e9aa7640f2a3d241242e73e1421f438e834c79 not found: ID does not exist" containerID="b5102643cb97930058c6eae715e9aa7640f2a3d241242e73e1421f438e834c79" Oct 29 23:34:46.843006 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.842894 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5102643cb97930058c6eae715e9aa7640f2a3d241242e73e1421f438e834c79"} err="failed to get container status \"b5102643cb97930058c6eae715e9aa7640f2a3d241242e73e1421f438e834c79\": rpc error: code = NotFound desc = could not find container \"b5102643cb97930058c6eae715e9aa7640f2a3d241242e73e1421f438e834c79\": container with ID starting with b5102643cb97930058c6eae715e9aa7640f2a3d241242e73e1421f438e834c79 not found: ID does not exist" Oct 29 23:34:46.861527 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.861463 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c103618d-ee4d-4568-bdaf-6e1b486f9277-maas-api-tls" pod "c103618d-ee4d-4568-bdaf-6e1b486f9277" (UID: "c103618d-ee4d-4568-bdaf-6e1b486f9277"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:34:46.889956 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.889927 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mv2dj\" (UniqueName: \"kubernetes.io/projected/c103618d-ee4d-4568-bdaf-6e1b486f9277-kube-api-access-mv2dj\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:46.889956 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.889954 5056 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c103618d-ee4d-4568-bdaf-6e1b486f9277-metrics-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:46.890138 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.889968 5056 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c103618d-ee4d-4568-bdaf-6e1b486f9277-maas-api-tls\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:46.890138 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.889979 5056 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c103618d-ee4d-4568-bdaf-6e1b486f9277-openshift-service-ca\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:46.939957 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:46.939907 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" podUID="4cb6d795-205b-465f-91b0-fa184fa8fe44" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 29 23:34:47.170583 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:47.170545 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7"] Oct 29 23:34:47.179149 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:47.179114 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-2a53c9-c65b576fb-k2fs7"] Oct 29 23:34:47.493329 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:47.493298 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:47.694854 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:47.683841 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:47.844336 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:47.841832 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:48.100808 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:48.100729 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:48.771383 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:48.771345 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c103618d-ee4d-4568-bdaf-6e1b486f9277" path="/var/lib/kubelet/pods/c103618d-ee4d-4568-bdaf-6e1b486f9277/volumes" Oct 29 23:34:49.193828 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.193799 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:49.473080 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.473016 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" Oct 29 23:34:49.534856 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.534828 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4cb6d795-205b-465f-91b0-fa184fa8fe44-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cb6d795-205b-465f-91b0-fa184fa8fe44-plugins-config-volume\") pod \"4cb6d795-205b-465f-91b0-fa184fa8fe44\" (UID: \"4cb6d795-205b-465f-91b0-fa184fa8fe44\") " Oct 29 23:34:49.535050 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.534874 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4cb6d795-205b-465f-91b0-fa184fa8fe44-kube-api-access-6h5qm\" (UniqueName: \"kubernetes.io/projected/4cb6d795-205b-465f-91b0-fa184fa8fe44-kube-api-access-6h5qm\") pod \"4cb6d795-205b-465f-91b0-fa184fa8fe44\" (UID: \"4cb6d795-205b-465f-91b0-fa184fa8fe44\") " Oct 29 23:34:49.535893 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.535866 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cb6d795-205b-465f-91b0-fa184fa8fe44-plugins-config-volume" pod "4cb6d795-205b-465f-91b0-fa184fa8fe44" (UID: "4cb6d795-205b-465f-91b0-fa184fa8fe44"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:34:49.538292 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.537789 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cb6d795-205b-465f-91b0-fa184fa8fe44-kube-api-access-6h5qm" pod "4cb6d795-205b-465f-91b0-fa184fa8fe44" (UID: "4cb6d795-205b-465f-91b0-fa184fa8fe44"). InnerVolumeSpecName "kube-api-access-6h5qm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:34:49.655076 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.655030 5056 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4cb6d795-205b-465f-91b0-fa184fa8fe44-plugins-config-volume\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:49.655372 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.655344 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6h5qm\" (UniqueName: \"kubernetes.io/projected/4cb6d795-205b-465f-91b0-fa184fa8fe44-kube-api-access-6h5qm\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:34:49.672465 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.662841 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:49.850318 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.850211 5056 generic.go:356] "Generic (PLEG): container finished" podID="4cb6d795-205b-465f-91b0-fa184fa8fe44" containerID="f9e75e3916a6ba4fcd6426fd98138674c80f81b5bdf4aad07ac2e244de5d0154" exitCode=0 Oct 29 23:34:49.850318 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.850249 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" event={"ID":"4cb6d795-205b-465f-91b0-fa184fa8fe44","Type":"ContainerDied","Data":"f9e75e3916a6ba4fcd6426fd98138674c80f81b5bdf4aad07ac2e244de5d0154"} Oct 29 23:34:49.850318 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.850284 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" Oct 29 23:34:49.850556 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.850324 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk" event={"ID":"4cb6d795-205b-465f-91b0-fa184fa8fe44","Type":"ContainerDied","Data":"9238524f85f6c9bcf44313444f7ae6bb5bbbc0178cdf3582d56bad1fe2f41199"} Oct 29 23:34:49.850556 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.850347 5056 scope.go:122] "RemoveContainer" containerID="f9e75e3916a6ba4fcd6426fd98138674c80f81b5bdf4aad07ac2e244de5d0154" Oct 29 23:34:49.860479 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.860456 5056 scope.go:122] "RemoveContainer" containerID="f9e75e3916a6ba4fcd6426fd98138674c80f81b5bdf4aad07ac2e244de5d0154" Oct 29 23:34:49.860811 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:34:49.860762 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9e75e3916a6ba4fcd6426fd98138674c80f81b5bdf4aad07ac2e244de5d0154\": container with ID starting with f9e75e3916a6ba4fcd6426fd98138674c80f81b5bdf4aad07ac2e244de5d0154 not found: ID does not exist" containerID="f9e75e3916a6ba4fcd6426fd98138674c80f81b5bdf4aad07ac2e244de5d0154" Oct 29 23:34:49.860904 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.860824 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9e75e3916a6ba4fcd6426fd98138674c80f81b5bdf4aad07ac2e244de5d0154"} err="failed to get container status \"f9e75e3916a6ba4fcd6426fd98138674c80f81b5bdf4aad07ac2e244de5d0154\": rpc error: code = NotFound desc = could not find container \"f9e75e3916a6ba4fcd6426fd98138674c80f81b5bdf4aad07ac2e244de5d0154\": container with ID starting with f9e75e3916a6ba4fcd6426fd98138674c80f81b5bdf4aad07ac2e244de5d0154 not found: ID does not exist" Oct 29 23:34:49.905686 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.905661 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk"] Oct 29 23:34:49.918481 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:49.918456 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-2a53c9-69bd69595-r6ttk"] Oct 29 23:34:50.770864 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:50.770830 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4cb6d795-205b-465f-91b0-fa184fa8fe44" path="/var/lib/kubelet/pods/4cb6d795-205b-465f-91b0-fa184fa8fe44/volumes" Oct 29 23:34:51.682184 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:51.676684 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:52.426860 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:34:52.426818 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:34:52.917294 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:52.917172 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:54.228297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:54.225557 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:54.969255 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:54.969222 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:55.133314 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:55.130639 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:55.553286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:55.544163 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:55.752282 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:55.750314 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj"] Oct 29 23:34:55.752282 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:55.750539 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" podUID="d8622875-ba05-4797-92ce-4e208a3a0540" containerName="istio-proxy" containerID="cri-o://8a1d5b171e3633570c48bdf16d25895e85583c9af83fc18bc32021075ed07b9a" gracePeriod=30 Oct 29 23:34:55.755905 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:55.753658 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv"] Oct 29 23:34:55.755905 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:55.753846 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" podUID="013d55e5-af92-4585-b7c4-33f76c3dabd4" containerName="istio-proxy" containerID="cri-o://b97ac2eb5afbe9a86eea78f9fbcaf132147b9152f3b2811fbb16bceec2843965" gracePeriod=30 Oct 29 23:34:55.767142 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:55.767110 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:55.767304 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:55.767197 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz"] Oct 29 23:34:55.768073 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:55.768043 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" podUID="29e6194d-41e0-4e53-8698-32ace257af0e" containerName="istio-proxy" containerID="cri-o://4afaa8cd928abf6e3b4a4d2448408e4892e9e46c3e36ce0bf11c7356d6b09b5a" gracePeriod=30 Oct 29 23:34:55.771969 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:55.771947 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h"] Oct 29 23:34:55.772156 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:55.772133 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" podUID="9369cf81-a466-4661-b370-e676e87abd4b" containerName="istio-proxy" containerID="cri-o://ef78b9dd06e4184263dfab3dfde4b1a32bd84adbb83c91009cc1365b3d986e2b" gracePeriod=30 Oct 29 23:34:55.952285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:55.949019 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:56.505716 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:56.505681 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:56.693203 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:56.693037 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:56.900478 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:56.900448 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:57.154097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:57.153998 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:57.336287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:57.333168 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:57.593780 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:57.593702 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:57.806384 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:57.806342 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk"] Oct 29 23:34:57.964347 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:57.964285 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:58.764975 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:58.764941 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:59.214735 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:59.214665 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:34:59.370609 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:34:59.370581 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:00.581470 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:00.581358 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:01.014417 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.014348 5056 generic.go:356] "Generic (PLEG): container finished" podID="013d55e5-af92-4585-b7c4-33f76c3dabd4" containerID="b97ac2eb5afbe9a86eea78f9fbcaf132147b9152f3b2811fbb16bceec2843965" exitCode=0 Oct 29 23:35:01.014561 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.014496 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" event={"ID":"013d55e5-af92-4585-b7c4-33f76c3dabd4","Type":"ContainerDied","Data":"b97ac2eb5afbe9a86eea78f9fbcaf132147b9152f3b2811fbb16bceec2843965"} Oct 29 23:35:01.016868 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.016845 5056 generic.go:356] "Generic (PLEG): container finished" podID="29e6194d-41e0-4e53-8698-32ace257af0e" containerID="4afaa8cd928abf6e3b4a4d2448408e4892e9e46c3e36ce0bf11c7356d6b09b5a" exitCode=0 Oct 29 23:35:01.016993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.016927 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" event={"ID":"29e6194d-41e0-4e53-8698-32ace257af0e","Type":"ContainerDied","Data":"4afaa8cd928abf6e3b4a4d2448408e4892e9e46c3e36ce0bf11c7356d6b09b5a"} Oct 29 23:35:01.016993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.016962 5056 scope.go:122] "RemoveContainer" containerID="4e2c81a74eb98b84f566faafb4f40f4148e0b7f4bb3c15b22a79f45d4998a2c1" Oct 29 23:35:01.020444 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.020422 5056 generic.go:356] "Generic (PLEG): container finished" podID="9369cf81-a466-4661-b370-e676e87abd4b" containerID="ef78b9dd06e4184263dfab3dfde4b1a32bd84adbb83c91009cc1365b3d986e2b" exitCode=0 Oct 29 23:35:01.020556 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.020527 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" event={"ID":"9369cf81-a466-4661-b370-e676e87abd4b","Type":"ContainerDied","Data":"ef78b9dd06e4184263dfab3dfde4b1a32bd84adbb83c91009cc1365b3d986e2b"} Oct 29 23:35:01.022514 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.022080 5056 generic.go:356] "Generic (PLEG): container finished" podID="d8622875-ba05-4797-92ce-4e208a3a0540" containerID="8a1d5b171e3633570c48bdf16d25895e85583c9af83fc18bc32021075ed07b9a" exitCode=0 Oct 29 23:35:01.022514 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.022155 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" event={"ID":"d8622875-ba05-4797-92ce-4e208a3a0540","Type":"ContainerDied","Data":"8a1d5b171e3633570c48bdf16d25895e85583c9af83fc18bc32021075ed07b9a"} Oct 29 23:35:01.081775 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.081742 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:35:01.120280 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.120242 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:35:01.155092 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.155052 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-token\" (UniqueName: \"kubernetes.io/projected/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-token\") pod \"013d55e5-af92-4585-b7c4-33f76c3dabd4\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " Oct 29 23:35:01.155299 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.155153 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-credential-socket\") pod \"013d55e5-af92-4585-b7c4-33f76c3dabd4\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " Oct 29 23:35:01.155299 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.155201 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/013d55e5-af92-4585-b7c4-33f76c3dabd4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/013d55e5-af92-4585-b7c4-33f76c3dabd4-istiod-ca-cert\") pod \"013d55e5-af92-4585-b7c4-33f76c3dabd4\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " Oct 29 23:35:01.155299 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.155238 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-data\") pod \"013d55e5-af92-4585-b7c4-33f76c3dabd4\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " Oct 29 23:35:01.155497 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.155310 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-envoy\") pod \"013d55e5-af92-4585-b7c4-33f76c3dabd4\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " Oct 29 23:35:01.155497 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.155364 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-workload-socket\") pod \"013d55e5-af92-4585-b7c4-33f76c3dabd4\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " Oct 29 23:35:01.155497 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.155391 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-podinfo\") pod \"013d55e5-af92-4585-b7c4-33f76c3dabd4\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " Oct 29 23:35:01.155497 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.155435 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-workload-certs\") pod \"013d55e5-af92-4585-b7c4-33f76c3dabd4\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " Oct 29 23:35:01.155715 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.155509 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/013d55e5-af92-4585-b7c4-33f76c3dabd4-kube-api-access-q57tj\" (UniqueName: \"kubernetes.io/projected/013d55e5-af92-4585-b7c4-33f76c3dabd4-kube-api-access-q57tj\") pod \"013d55e5-af92-4585-b7c4-33f76c3dabd4\" (UID: \"013d55e5-af92-4585-b7c4-33f76c3dabd4\") " Oct 29 23:35:01.156575 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.156540 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-data" pod "013d55e5-af92-4585-b7c4-33f76c3dabd4" (UID: "013d55e5-af92-4585-b7c4-33f76c3dabd4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.156704 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.156684 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-workload-socket" pod "013d55e5-af92-4585-b7c4-33f76c3dabd4" (UID: "013d55e5-af92-4585-b7c4-33f76c3dabd4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.156777 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.156703 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-credential-socket" pod "013d55e5-af92-4585-b7c4-33f76c3dabd4" (UID: "013d55e5-af92-4585-b7c4-33f76c3dabd4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.157044 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.156852 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-workload-certs" pod "013d55e5-af92-4585-b7c4-33f76c3dabd4" (UID: "013d55e5-af92-4585-b7c4-33f76c3dabd4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.157044 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.157008 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/013d55e5-af92-4585-b7c4-33f76c3dabd4-istiod-ca-cert" pod "013d55e5-af92-4585-b7c4-33f76c3dabd4" (UID: "013d55e5-af92-4585-b7c4-33f76c3dabd4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:01.158915 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.158879 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-token" pod "013d55e5-af92-4585-b7c4-33f76c3dabd4" (UID: "013d55e5-af92-4585-b7c4-33f76c3dabd4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:01.159322 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.159297 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/013d55e5-af92-4585-b7c4-33f76c3dabd4-kube-api-access-q57tj" pod "013d55e5-af92-4585-b7c4-33f76c3dabd4" (UID: "013d55e5-af92-4585-b7c4-33f76c3dabd4"). InnerVolumeSpecName "kube-api-access-q57tj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:01.159402 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.159341 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-podinfo" pod "013d55e5-af92-4585-b7c4-33f76c3dabd4" (UID: "013d55e5-af92-4585-b7c4-33f76c3dabd4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:35:01.160580 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.160556 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-envoy" pod "013d55e5-af92-4585-b7c4-33f76c3dabd4" (UID: "013d55e5-af92-4585-b7c4-33f76c3dabd4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.180547 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.180520 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:35:01.256726 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.256691 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9369cf81-a466-4661-b370-e676e87abd4b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9369cf81-a466-4661-b370-e676e87abd4b-istiod-ca-cert\") pod \"9369cf81-a466-4661-b370-e676e87abd4b\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " Oct 29 23:35:01.256916 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.256761 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-workload-certs\") pod \"29e6194d-41e0-4e53-8698-32ace257af0e\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " Oct 29 23:35:01.256916 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.256795 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-credential-socket\") pod \"29e6194d-41e0-4e53-8698-32ace257af0e\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " Oct 29 23:35:01.256916 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.256821 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-workload-socket\") pod \"29e6194d-41e0-4e53-8698-32ace257af0e\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " Oct 29 23:35:01.256916 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.256885 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/29e6194d-41e0-4e53-8698-32ace257af0e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29e6194d-41e0-4e53-8698-32ace257af0e-istio-podinfo\") pod \"29e6194d-41e0-4e53-8698-32ace257af0e\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " Oct 29 23:35:01.257125 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.256917 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29e6194d-41e0-4e53-8698-32ace257af0e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29e6194d-41e0-4e53-8698-32ace257af0e-istiod-ca-cert\") pod \"29e6194d-41e0-4e53-8698-32ace257af0e\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " Oct 29 23:35:01.257125 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.256960 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9369cf81-a466-4661-b370-e676e87abd4b-istio-token\" (UniqueName: \"kubernetes.io/projected/9369cf81-a466-4661-b370-e676e87abd4b-istio-token\") pod \"9369cf81-a466-4661-b370-e676e87abd4b\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " Oct 29 23:35:01.257125 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.257053 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29e6194d-41e0-4e53-8698-32ace257af0e-istio-token\" (UniqueName: \"kubernetes.io/projected/29e6194d-41e0-4e53-8698-32ace257af0e-istio-token\") pod \"29e6194d-41e0-4e53-8698-32ace257af0e\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " Oct 29 23:35:01.257125 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.257102 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-credential-socket\") pod \"9369cf81-a466-4661-b370-e676e87abd4b\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " Oct 29 23:35:01.257344 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.257170 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9369cf81-a466-4661-b370-e676e87abd4b-kube-api-access-fd4bz\" (UniqueName: \"kubernetes.io/projected/9369cf81-a466-4661-b370-e676e87abd4b-kube-api-access-fd4bz\") pod \"9369cf81-a466-4661-b370-e676e87abd4b\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " Oct 29 23:35:01.257344 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.257214 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-istio-envoy\") pod \"9369cf81-a466-4661-b370-e676e87abd4b\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " Oct 29 23:35:01.257344 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.257251 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-istio-envoy\") pod \"29e6194d-41e0-4e53-8698-32ace257af0e\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " Oct 29 23:35:01.257344 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.257317 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-istio-data\") pod \"29e6194d-41e0-4e53-8698-32ace257af0e\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " Oct 29 23:35:01.257547 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.257400 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-istio-data\") pod \"9369cf81-a466-4661-b370-e676e87abd4b\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " Oct 29 23:35:01.257547 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.257494 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9369cf81-a466-4661-b370-e676e87abd4b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9369cf81-a466-4661-b370-e676e87abd4b-istio-podinfo\") pod \"9369cf81-a466-4661-b370-e676e87abd4b\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " Oct 29 23:35:01.257647 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.257560 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-workload-certs\") pod \"9369cf81-a466-4661-b370-e676e87abd4b\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " Oct 29 23:35:01.257647 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.257600 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-workload-socket\") pod \"9369cf81-a466-4661-b370-e676e87abd4b\" (UID: \"9369cf81-a466-4661-b370-e676e87abd4b\") " Oct 29 23:35:01.257745 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.257656 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29e6194d-41e0-4e53-8698-32ace257af0e-kube-api-access-wkpvv\" (UniqueName: \"kubernetes.io/projected/29e6194d-41e0-4e53-8698-32ace257af0e-kube-api-access-wkpvv\") pod \"29e6194d-41e0-4e53-8698-32ace257af0e\" (UID: \"29e6194d-41e0-4e53-8698-32ace257af0e\") " Oct 29 23:35:01.257745 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.257706 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29e6194d-41e0-4e53-8698-32ace257af0e-istiod-ca-cert" pod "29e6194d-41e0-4e53-8698-32ace257af0e" (UID: "29e6194d-41e0-4e53-8698-32ace257af0e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:01.258108 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.258083 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9369cf81-a466-4661-b370-e676e87abd4b-istiod-ca-cert" pod "9369cf81-a466-4661-b370-e676e87abd4b" (UID: "9369cf81-a466-4661-b370-e676e87abd4b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:01.258341 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.258318 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.258438 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.258346 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.258438 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.258362 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.258438 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.258374 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q57tj\" (UniqueName: \"kubernetes.io/projected/013d55e5-af92-4585-b7c4-33f76c3dabd4-kube-api-access-q57tj\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.258438 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.258403 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9369cf81-a466-4661-b370-e676e87abd4b-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.258438 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.258418 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29e6194d-41e0-4e53-8698-32ace257af0e-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.258438 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.258431 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.258742 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.258445 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.258742 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.258459 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/013d55e5-af92-4585-b7c4-33f76c3dabd4-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.258742 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.258474 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.258742 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.258488 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/013d55e5-af92-4585-b7c4-33f76c3dabd4-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.258944 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.258761 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-workload-socket" pod "29e6194d-41e0-4e53-8698-32ace257af0e" (UID: "29e6194d-41e0-4e53-8698-32ace257af0e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.259490 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.259459 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-istio-data" pod "29e6194d-41e0-4e53-8698-32ace257af0e" (UID: "29e6194d-41e0-4e53-8698-32ace257af0e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.260661 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.259768 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-credential-socket" pod "9369cf81-a466-4661-b370-e676e87abd4b" (UID: "9369cf81-a466-4661-b370-e676e87abd4b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.260661 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.260080 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-workload-certs" pod "29e6194d-41e0-4e53-8698-32ace257af0e" (UID: "29e6194d-41e0-4e53-8698-32ace257af0e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.260661 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.260356 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-workload-certs" pod "9369cf81-a466-4661-b370-e676e87abd4b" (UID: "9369cf81-a466-4661-b370-e676e87abd4b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.261200 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.261179 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-credential-socket" pod "29e6194d-41e0-4e53-8698-32ace257af0e" (UID: "29e6194d-41e0-4e53-8698-32ace257af0e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.262993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.262583 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9369cf81-a466-4661-b370-e676e87abd4b-kube-api-access-fd4bz" pod "9369cf81-a466-4661-b370-e676e87abd4b" (UID: "9369cf81-a466-4661-b370-e676e87abd4b"). InnerVolumeSpecName "kube-api-access-fd4bz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:01.262993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.262794 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-workload-socket" pod "9369cf81-a466-4661-b370-e676e87abd4b" (UID: "9369cf81-a466-4661-b370-e676e87abd4b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.262993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.262961 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-istio-data" pod "9369cf81-a466-4661-b370-e676e87abd4b" (UID: "9369cf81-a466-4661-b370-e676e87abd4b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.263619 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.263596 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-istio-envoy" pod "9369cf81-a466-4661-b370-e676e87abd4b" (UID: "9369cf81-a466-4661-b370-e676e87abd4b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.264616 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.264583 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29e6194d-41e0-4e53-8698-32ace257af0e-istio-token" pod "29e6194d-41e0-4e53-8698-32ace257af0e" (UID: "29e6194d-41e0-4e53-8698-32ace257af0e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:01.264716 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.264624 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9369cf81-a466-4661-b370-e676e87abd4b-istio-podinfo" pod "9369cf81-a466-4661-b370-e676e87abd4b" (UID: "9369cf81-a466-4661-b370-e676e87abd4b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:35:01.265237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.265214 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-istio-envoy" pod "29e6194d-41e0-4e53-8698-32ace257af0e" (UID: "29e6194d-41e0-4e53-8698-32ace257af0e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.265650 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.265630 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29e6194d-41e0-4e53-8698-32ace257af0e-kube-api-access-wkpvv" pod "29e6194d-41e0-4e53-8698-32ace257af0e" (UID: "29e6194d-41e0-4e53-8698-32ace257af0e"). InnerVolumeSpecName "kube-api-access-wkpvv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:01.266492 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.266465 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9369cf81-a466-4661-b370-e676e87abd4b-istio-token" pod "9369cf81-a466-4661-b370-e676e87abd4b" (UID: "9369cf81-a466-4661-b370-e676e87abd4b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:01.267181 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.267156 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29e6194d-41e0-4e53-8698-32ace257af0e-istio-podinfo" pod "29e6194d-41e0-4e53-8698-32ace257af0e" (UID: "29e6194d-41e0-4e53-8698-32ace257af0e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:35:01.360757 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.360676 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.360757 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.360716 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.360757 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.360732 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.360757 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.360746 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29e6194d-41e0-4e53-8698-32ace257af0e-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.361133 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.360764 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9369cf81-a466-4661-b370-e676e87abd4b-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.361133 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.360777 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29e6194d-41e0-4e53-8698-32ace257af0e-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.361133 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.360791 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.361133 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.360805 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fd4bz\" (UniqueName: \"kubernetes.io/projected/9369cf81-a466-4661-b370-e676e87abd4b-kube-api-access-fd4bz\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.361133 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.360820 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.361133 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.360835 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.361133 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.360849 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29e6194d-41e0-4e53-8698-32ace257af0e-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.361133 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.360862 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.361133 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.360876 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9369cf81-a466-4661-b370-e676e87abd4b-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.361133 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.360892 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.361133 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.360905 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9369cf81-a466-4661-b370-e676e87abd4b-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.361133 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.360919 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkpvv\" (UniqueName: \"kubernetes.io/projected/29e6194d-41e0-4e53-8698-32ace257af0e-kube-api-access-wkpvv\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.427633 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.427607 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:35:01.563551 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.563514 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-workload-certs\") pod \"d8622875-ba05-4797-92ce-4e208a3a0540\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " Oct 29 23:35:01.563551 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.563558 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8622875-ba05-4797-92ce-4e208a3a0540-istio-token\" (UniqueName: \"kubernetes.io/projected/d8622875-ba05-4797-92ce-4e208a3a0540-istio-token\") pod \"d8622875-ba05-4797-92ce-4e208a3a0540\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " Oct 29 23:35:01.563795 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.563595 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-istio-data\") pod \"d8622875-ba05-4797-92ce-4e208a3a0540\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " Oct 29 23:35:01.563795 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.563622 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d8622875-ba05-4797-92ce-4e208a3a0540-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8622875-ba05-4797-92ce-4e208a3a0540-istio-podinfo\") pod \"d8622875-ba05-4797-92ce-4e208a3a0540\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " Oct 29 23:35:01.563913 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.563793 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-credential-socket\") pod \"d8622875-ba05-4797-92ce-4e208a3a0540\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " Oct 29 23:35:01.563913 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.563846 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-workload-certs" pod "d8622875-ba05-4797-92ce-4e208a3a0540" (UID: "d8622875-ba05-4797-92ce-4e208a3a0540"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.563913 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.563882 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d8622875-ba05-4797-92ce-4e208a3a0540-kube-api-access-4gldx\" (UniqueName: \"kubernetes.io/projected/d8622875-ba05-4797-92ce-4e208a3a0540-kube-api-access-4gldx\") pod \"d8622875-ba05-4797-92ce-4e208a3a0540\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " Oct 29 23:35:01.564072 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.563881 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-istio-data" pod "d8622875-ba05-4797-92ce-4e208a3a0540" (UID: "d8622875-ba05-4797-92ce-4e208a3a0540"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.564134 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.564095 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-credential-socket" pod "d8622875-ba05-4797-92ce-4e208a3a0540" (UID: "d8622875-ba05-4797-92ce-4e208a3a0540"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.564196 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.564159 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-istio-envoy\") pod \"d8622875-ba05-4797-92ce-4e208a3a0540\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " Oct 29 23:35:01.564243 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.564190 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-workload-socket\") pod \"d8622875-ba05-4797-92ce-4e208a3a0540\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " Oct 29 23:35:01.564243 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.564237 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d8622875-ba05-4797-92ce-4e208a3a0540-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8622875-ba05-4797-92ce-4e208a3a0540-istiod-ca-cert\") pod \"d8622875-ba05-4797-92ce-4e208a3a0540\" (UID: \"d8622875-ba05-4797-92ce-4e208a3a0540\") " Oct 29 23:35:01.564495 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.564470 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-workload-socket" pod "d8622875-ba05-4797-92ce-4e208a3a0540" (UID: "d8622875-ba05-4797-92ce-4e208a3a0540"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.564840 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.564816 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d8622875-ba05-4797-92ce-4e208a3a0540-istiod-ca-cert" pod "d8622875-ba05-4797-92ce-4e208a3a0540" (UID: "d8622875-ba05-4797-92ce-4e208a3a0540"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:01.564998 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.564980 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.565067 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.565004 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.565067 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.565020 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d8622875-ba05-4797-92ce-4e208a3a0540-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.565067 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.565033 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.565067 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.565046 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.566107 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.566082 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8622875-ba05-4797-92ce-4e208a3a0540-istio-token" pod "d8622875-ba05-4797-92ce-4e208a3a0540" (UID: "d8622875-ba05-4797-92ce-4e208a3a0540"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:01.566208 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.566103 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d8622875-ba05-4797-92ce-4e208a3a0540-istio-podinfo" pod "d8622875-ba05-4797-92ce-4e208a3a0540" (UID: "d8622875-ba05-4797-92ce-4e208a3a0540"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:35:01.567290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.567270 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-istio-envoy" pod "d8622875-ba05-4797-92ce-4e208a3a0540" (UID: "d8622875-ba05-4797-92ce-4e208a3a0540"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:01.567361 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.567320 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8622875-ba05-4797-92ce-4e208a3a0540-kube-api-access-4gldx" pod "d8622875-ba05-4797-92ce-4e208a3a0540" (UID: "d8622875-ba05-4797-92ce-4e208a3a0540"). InnerVolumeSpecName "kube-api-access-4gldx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:01.666374 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.666338 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d8622875-ba05-4797-92ce-4e208a3a0540-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.666374 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.666369 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d8622875-ba05-4797-92ce-4e208a3a0540-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.666374 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.666380 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d8622875-ba05-4797-92ce-4e208a3a0540-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:01.666819 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:01.666389 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gldx\" (UniqueName: \"kubernetes.io/projected/d8622875-ba05-4797-92ce-4e208a3a0540-kube-api-access-4gldx\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:02.032054 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.031970 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" Oct 29 23:35:02.032234 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.031970 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj" event={"ID":"d8622875-ba05-4797-92ce-4e208a3a0540","Type":"ContainerDied","Data":"4a45efbf7f2fdc5b7567cd132bb2218815ac861f04619083abdf733cf21473c1"} Oct 29 23:35:02.032234 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.032133 5056 scope.go:122] "RemoveContainer" containerID="8a1d5b171e3633570c48bdf16d25895e85583c9af83fc18bc32021075ed07b9a" Oct 29 23:35:02.033718 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.033698 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" Oct 29 23:35:02.033841 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.033718 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv" event={"ID":"013d55e5-af92-4585-b7c4-33f76c3dabd4","Type":"ContainerDied","Data":"51415334dc3241cc7f2df8ab2c2c85b8eee23becd3d12817d16bf602b5efe12d"} Oct 29 23:35:02.036151 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.035430 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" event={"ID":"29e6194d-41e0-4e53-8698-32ace257af0e","Type":"ContainerDied","Data":"ca8f504f55b17b645aa60ea0fdc8654b105c86da58088775201c99eafba72433"} Oct 29 23:35:02.036151 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.035443 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz" Oct 29 23:35:02.040730 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.038688 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" event={"ID":"9369cf81-a466-4661-b370-e676e87abd4b","Type":"ContainerDied","Data":"7b7f201b74b077822d90ba1951d02a5b34dcf4b87e736919b381cdb7c5d7cbdd"} Oct 29 23:35:02.040730 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.038794 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h" Oct 29 23:35:02.058067 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.058045 5056 scope.go:122] "RemoveContainer" containerID="b97ac2eb5afbe9a86eea78f9fbcaf132147b9152f3b2811fbb16bceec2843965" Oct 29 23:35:02.070009 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.069986 5056 scope.go:122] "RemoveContainer" containerID="4afaa8cd928abf6e3b4a4d2448408e4892e9e46c3e36ce0bf11c7356d6b09b5a" Oct 29 23:35:02.079447 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.079394 5056 scope.go:122] "RemoveContainer" containerID="ef78b9dd06e4184263dfab3dfde4b1a32bd84adbb83c91009cc1365b3d986e2b" Oct 29 23:35:02.137114 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.137078 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj"] Oct 29 23:35:02.150271 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.150226 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-x55zj"] Oct 29 23:35:02.187541 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.187507 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv"] Oct 29 23:35:02.192053 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.192027 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-7fgtv"] Oct 29 23:35:02.230402 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.230368 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz"] Oct 29 23:35:02.233000 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.232974 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-qf6hz"] Oct 29 23:35:02.271903 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.271871 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h"] Oct 29 23:35:02.280904 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.280878 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-2a53c9-openshift-default-6fdbfb78c7-jsl5h"] Oct 29 23:35:02.453214 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.453179 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:02.772416 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.772341 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="013d55e5-af92-4585-b7c4-33f76c3dabd4" path="/var/lib/kubelet/pods/013d55e5-af92-4585-b7c4-33f76c3dabd4/volumes" Oct 29 23:35:02.772874 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.772859 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29e6194d-41e0-4e53-8698-32ace257af0e" path="/var/lib/kubelet/pods/29e6194d-41e0-4e53-8698-32ace257af0e/volumes" Oct 29 23:35:02.773316 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.773300 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9369cf81-a466-4661-b370-e676e87abd4b" path="/var/lib/kubelet/pods/9369cf81-a466-4661-b370-e676e87abd4b/volumes" Oct 29 23:35:02.774049 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:02.774034 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d8622875-ba05-4797-92ce-4e208a3a0540" path="/var/lib/kubelet/pods/d8622875-ba05-4797-92ce-4e208a3a0540/volumes" Oct 29 23:35:03.733422 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:03.733388 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:03.889626 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:03.889589 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:04.667235 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:04.667202 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:04.824638 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:04.824605 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:06.965548 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:06.965472 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:08.622773 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:08.622726 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:09.268530 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:09.268477 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:09.456891 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:09.456847 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:09.946703 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:09.946664 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:11.319528 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:11.319497 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:12.017665 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:12.017630 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:12.827061 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:12.827020 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" podUID="da79ef4d-119b-4eed-870b-5e0fb48e60c8" containerName="main" containerID="cri-o://f19887b1c92f9a8231ed13b29c82cf9a1afed849ddd343bcf3cbbc06961bca5e" gracePeriod=45 Oct 29 23:35:12.992149 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:12.992106 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" podUID="da79ef4d-119b-4eed-870b-5e0fb48e60c8" containerName="main" probeResult="failure" output="Get \"https://10.132.0.88:8000/ready\": dial tcp 10.132.0.88:8000: connect: connection refused" Oct 29 23:35:13.101161 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.101132 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:35:13.147954 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.147926 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/da79ef4d-119b-4eed-870b-5e0fb48e60c8-kube-api-access-g46gd\" (UniqueName: \"kubernetes.io/projected/da79ef4d-119b-4eed-870b-5e0fb48e60c8-kube-api-access-g46gd\") pod \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " Oct 29 23:35:13.148136 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.147971 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-model-cache\") pod \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " Oct 29 23:35:13.148136 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.148032 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-tmp-dir\") pod \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " Oct 29 23:35:13.148136 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.148123 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-dshm\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-dshm\") pod \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " Oct 29 23:35:13.148319 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.148303 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/da79ef4d-119b-4eed-870b-5e0fb48e60c8-tls-certs\" (UniqueName: \"kubernetes.io/secret/da79ef4d-119b-4eed-870b-5e0fb48e60c8-tls-certs\") pod \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " Oct 29 23:35:13.148367 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.148314 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-model-cache" pod "da79ef4d-119b-4eed-870b-5e0fb48e60c8" (UID: "da79ef4d-119b-4eed-870b-5e0fb48e60c8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:13.148367 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.148356 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-home\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-home\") pod \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\" (UID: \"da79ef4d-119b-4eed-870b-5e0fb48e60c8\") " Oct 29 23:35:13.148469 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.148394 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-tmp-dir" pod "da79ef4d-119b-4eed-870b-5e0fb48e60c8" (UID: "da79ef4d-119b-4eed-870b-5e0fb48e60c8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:13.148596 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.148569 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-home" pod "da79ef4d-119b-4eed-870b-5e0fb48e60c8" (UID: "da79ef4d-119b-4eed-870b-5e0fb48e60c8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:13.149061 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.149003 5056 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-model-cache\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:13.149061 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.149027 5056 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-tmp-dir\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:13.149061 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.149040 5056 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-home\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:13.150675 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.150643 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-dshm" pod "da79ef4d-119b-4eed-870b-5e0fb48e60c8" (UID: "da79ef4d-119b-4eed-870b-5e0fb48e60c8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:13.150919 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.150889 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da79ef4d-119b-4eed-870b-5e0fb48e60c8-tls-certs" pod "da79ef4d-119b-4eed-870b-5e0fb48e60c8" (UID: "da79ef4d-119b-4eed-870b-5e0fb48e60c8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:35:13.151338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.151014 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da79ef4d-119b-4eed-870b-5e0fb48e60c8-kube-api-access-g46gd" pod "da79ef4d-119b-4eed-870b-5e0fb48e60c8" (UID: "da79ef4d-119b-4eed-870b-5e0fb48e60c8"). InnerVolumeSpecName "kube-api-access-g46gd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:13.182212 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.182185 5056 generic.go:356] "Generic (PLEG): container finished" podID="da79ef4d-119b-4eed-870b-5e0fb48e60c8" containerID="f19887b1c92f9a8231ed13b29c82cf9a1afed849ddd343bcf3cbbc06961bca5e" exitCode=0 Oct 29 23:35:13.182386 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.182372 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" event={"ID":"da79ef4d-119b-4eed-870b-5e0fb48e60c8","Type":"ContainerDied","Data":"f19887b1c92f9a8231ed13b29c82cf9a1afed849ddd343bcf3cbbc06961bca5e"} Oct 29 23:35:13.182444 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.182404 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" event={"ID":"da79ef4d-119b-4eed-870b-5e0fb48e60c8","Type":"ContainerDied","Data":"d080530a811ae92158ef19b35bc9948571723cb2f689a288f96ed484e2c398bc"} Oct 29 23:35:13.182444 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.182423 5056 scope.go:122] "RemoveContainer" containerID="f19887b1c92f9a8231ed13b29c82cf9a1afed849ddd343bcf3cbbc06961bca5e" Oct 29 23:35:13.182543 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.182533 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk" Oct 29 23:35:13.203424 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.203378 5056 scope.go:122] "RemoveContainer" containerID="f19887b1c92f9a8231ed13b29c82cf9a1afed849ddd343bcf3cbbc06961bca5e" Oct 29 23:35:13.203741 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:35:13.203720 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f19887b1c92f9a8231ed13b29c82cf9a1afed849ddd343bcf3cbbc06961bca5e\": container with ID starting with f19887b1c92f9a8231ed13b29c82cf9a1afed849ddd343bcf3cbbc06961bca5e not found: ID does not exist" containerID="f19887b1c92f9a8231ed13b29c82cf9a1afed849ddd343bcf3cbbc06961bca5e" Oct 29 23:35:13.203812 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.203750 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f19887b1c92f9a8231ed13b29c82cf9a1afed849ddd343bcf3cbbc06961bca5e"} err="failed to get container status \"f19887b1c92f9a8231ed13b29c82cf9a1afed849ddd343bcf3cbbc06961bca5e\": rpc error: code = NotFound desc = could not find container \"f19887b1c92f9a8231ed13b29c82cf9a1afed849ddd343bcf3cbbc06961bca5e\": container with ID starting with f19887b1c92f9a8231ed13b29c82cf9a1afed849ddd343bcf3cbbc06961bca5e not found: ID does not exist" Oct 29 23:35:13.249753 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.249712 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk"] Oct 29 23:35:13.253291 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.251692 5056 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/da79ef4d-119b-4eed-870b-5e0fb48e60c8-dshm\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:13.253291 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.251720 5056 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/da79ef4d-119b-4eed-870b-5e0fb48e60c8-tls-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:13.253291 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.251732 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g46gd\" (UniqueName: \"kubernetes.io/projected/da79ef4d-119b-4eed-870b-5e0fb48e60c8-kube-api-access-g46gd\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:13.255592 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.255567 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-ae1784/distinct-2-w6-ae1784-kserve-6f896cf7bb-xrzrk"] Oct 29 23:35:13.957285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:13.957101 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:14.770582 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:14.770546 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="da79ef4d-119b-4eed-870b-5e0fb48e60c8" path="/var/lib/kubelet/pods/da79ef4d-119b-4eed-870b-5e0fb48e60c8/volumes" Oct 29 23:35:15.792286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:15.791612 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:16.371790 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:16.371751 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:17.219840 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:17.219754 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:17.402075 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:17.402040 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:17.498469 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:17.497865 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc"] Oct 29 23:35:17.499511 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:17.499249 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" podUID="9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" containerName="istio-proxy" containerID="cri-o://9f3d57123fab029cee58404e9328eb28102fb843df4e1b141f42fa2f264f46d3" gracePeriod=30 Oct 29 23:35:17.516937 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:17.516901 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt"] Oct 29 23:35:17.517089 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:17.517031 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w"] Oct 29 23:35:17.517282 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:17.517223 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" podUID="b6e9182f-276e-4ca6-bf97-16a1cf21ba00" containerName="istio-proxy" containerID="cri-o://1fa9b3b5e91f9925de87dc7bf346c65a6825f6462ae86ef37ddf1cb9687de5b9" gracePeriod=30 Oct 29 23:35:17.517545 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:17.517520 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" podUID="7e1dad7e-beb0-49c7-be70-d99766042790" containerName="istio-proxy" containerID="cri-o://de2fd2b39bb6ebdc532019edc3e4712feb3c6c93a04f5d79c5748808349dce87" gracePeriod=30 Oct 29 23:35:17.527041 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:17.527007 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v"] Oct 29 23:35:17.527229 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:17.527208 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" podUID="1a24e9f0-98ce-4367-99b2-25653f0171ee" containerName="istio-proxy" containerID="cri-o://33da25dafe61f4d9fe53fd9107ffad5b16ffdf5ad21448598f3af69d1da459ff" gracePeriod=30 Oct 29 23:35:17.622280 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:17.616099 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:17.786649 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:17.786555 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:18.779243 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:18.779205 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:35:18.779636 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:18.779296 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" podUID="7e1dad7e-beb0-49c7-be70-d99766042790" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:35:18.889316 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:18.889271 5056 patch_prober.go:29] interesting pod/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:35:18.889497 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:18.889354 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" podUID="b6e9182f-276e-4ca6-bf97-16a1cf21ba00" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:35:19.803279 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:19.801947 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:21.553840 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:21.553795 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:22.138733 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:22.138698 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:22.298990 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:22.298952 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:22.416473 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:35:22.416382 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:35:22.995516 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:22.995163 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:35:23.091324 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.090726 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-istio-envoy\") pod \"7e1dad7e-beb0-49c7-be70-d99766042790\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " Oct 29 23:35:23.091324 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.090769 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7e1dad7e-beb0-49c7-be70-d99766042790-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e1dad7e-beb0-49c7-be70-d99766042790-istio-podinfo\") pod \"7e1dad7e-beb0-49c7-be70-d99766042790\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " Oct 29 23:35:23.091324 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.090804 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-credential-socket\") pod \"7e1dad7e-beb0-49c7-be70-d99766042790\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " Oct 29 23:35:23.091324 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.090862 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-workload-socket\") pod \"7e1dad7e-beb0-49c7-be70-d99766042790\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " Oct 29 23:35:23.091324 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.090904 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e1dad7e-beb0-49c7-be70-d99766042790-istio-token\" (UniqueName: \"kubernetes.io/projected/7e1dad7e-beb0-49c7-be70-d99766042790-istio-token\") pod \"7e1dad7e-beb0-49c7-be70-d99766042790\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " Oct 29 23:35:23.091324 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.090995 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e1dad7e-beb0-49c7-be70-d99766042790-kube-api-access-7h2p4\" (UniqueName: \"kubernetes.io/projected/7e1dad7e-beb0-49c7-be70-d99766042790-kube-api-access-7h2p4\") pod \"7e1dad7e-beb0-49c7-be70-d99766042790\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " Oct 29 23:35:23.091324 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.091037 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-istio-data\") pod \"7e1dad7e-beb0-49c7-be70-d99766042790\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " Oct 29 23:35:23.091324 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.091109 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-workload-certs\") pod \"7e1dad7e-beb0-49c7-be70-d99766042790\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " Oct 29 23:35:23.091324 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.091190 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e1dad7e-beb0-49c7-be70-d99766042790-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e1dad7e-beb0-49c7-be70-d99766042790-istiod-ca-cert\") pod \"7e1dad7e-beb0-49c7-be70-d99766042790\" (UID: \"7e1dad7e-beb0-49c7-be70-d99766042790\") " Oct 29 23:35:23.094206 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.092703 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-credential-socket" pod "7e1dad7e-beb0-49c7-be70-d99766042790" (UID: "7e1dad7e-beb0-49c7-be70-d99766042790"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.094206 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.093519 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-istio-data" pod "7e1dad7e-beb0-49c7-be70-d99766042790" (UID: "7e1dad7e-beb0-49c7-be70-d99766042790"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.094206 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.093692 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-workload-certs" pod "7e1dad7e-beb0-49c7-be70-d99766042790" (UID: "7e1dad7e-beb0-49c7-be70-d99766042790"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.094206 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.094148 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-workload-socket" pod "7e1dad7e-beb0-49c7-be70-d99766042790" (UID: "7e1dad7e-beb0-49c7-be70-d99766042790"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.095543 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.094878 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e1dad7e-beb0-49c7-be70-d99766042790-istiod-ca-cert" pod "7e1dad7e-beb0-49c7-be70-d99766042790" (UID: "7e1dad7e-beb0-49c7-be70-d99766042790"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:23.096816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.096735 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.096816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.096757 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e1dad7e-beb0-49c7-be70-d99766042790-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.096816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.096766 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.096816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.096774 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.096816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.096782 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.098018 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.097765 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e1dad7e-beb0-49c7-be70-d99766042790-istio-podinfo" pod "7e1dad7e-beb0-49c7-be70-d99766042790" (UID: "7e1dad7e-beb0-49c7-be70-d99766042790"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:35:23.101703 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.100206 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e1dad7e-beb0-49c7-be70-d99766042790-istio-token" pod "7e1dad7e-beb0-49c7-be70-d99766042790" (UID: "7e1dad7e-beb0-49c7-be70-d99766042790"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:23.106407 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.106355 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-istio-envoy" pod "7e1dad7e-beb0-49c7-be70-d99766042790" (UID: "7e1dad7e-beb0-49c7-be70-d99766042790"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.107880 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.107858 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:35:23.162287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.162199 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e1dad7e-beb0-49c7-be70-d99766042790-kube-api-access-7h2p4" pod "7e1dad7e-beb0-49c7-be70-d99766042790" (UID: "7e1dad7e-beb0-49c7-be70-d99766042790"). InnerVolumeSpecName "kube-api-access-7h2p4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:23.196188 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.195273 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql"] Oct 29 23:35:23.196188 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.195522 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" podUID="e80bef80-0024-4113-b7f5-f43eded6d18e" containerName="maas-api" containerID="cri-o://50637988903da9eee37952b4da1fbccdae355252ceb4ea02e87434770f457c03" gracePeriod=30 Oct 29 23:35:23.198935 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.197558 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-token\" (UniqueName: \"kubernetes.io/projected/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-token\") pod \"1a24e9f0-98ce-4367-99b2-25653f0171ee\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " Oct 29 23:35:23.198935 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.197619 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a24e9f0-98ce-4367-99b2-25653f0171ee-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a24e9f0-98ce-4367-99b2-25653f0171ee-istiod-ca-cert\") pod \"1a24e9f0-98ce-4367-99b2-25653f0171ee\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " Oct 29 23:35:23.198935 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.197660 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-workload-certs\") pod \"1a24e9f0-98ce-4367-99b2-25653f0171ee\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " Oct 29 23:35:23.198935 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.197702 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-envoy\") pod \"1a24e9f0-98ce-4367-99b2-25653f0171ee\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " Oct 29 23:35:23.198935 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.197753 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-workload-socket\") pod \"1a24e9f0-98ce-4367-99b2-25653f0171ee\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " Oct 29 23:35:23.198935 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.197803 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-podinfo\") pod \"1a24e9f0-98ce-4367-99b2-25653f0171ee\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " Oct 29 23:35:23.198935 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.197939 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a24e9f0-98ce-4367-99b2-25653f0171ee-kube-api-access-plgnk\" (UniqueName: \"kubernetes.io/projected/1a24e9f0-98ce-4367-99b2-25653f0171ee-kube-api-access-plgnk\") pod \"1a24e9f0-98ce-4367-99b2-25653f0171ee\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " Oct 29 23:35:23.198935 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.197975 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-data\") pod \"1a24e9f0-98ce-4367-99b2-25653f0171ee\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " Oct 29 23:35:23.198935 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.198053 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-credential-socket\") pod \"1a24e9f0-98ce-4367-99b2-25653f0171ee\" (UID: \"1a24e9f0-98ce-4367-99b2-25653f0171ee\") " Oct 29 23:35:23.198935 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.198641 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e1dad7e-beb0-49c7-be70-d99766042790-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.198935 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.198664 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e1dad7e-beb0-49c7-be70-d99766042790-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.198935 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.198679 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e1dad7e-beb0-49c7-be70-d99766042790-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.198935 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.198692 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h2p4\" (UniqueName: \"kubernetes.io/projected/7e1dad7e-beb0-49c7-be70-d99766042790-kube-api-access-7h2p4\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.200980 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.200672 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a24e9f0-98ce-4367-99b2-25653f0171ee-istiod-ca-cert" pod "1a24e9f0-98ce-4367-99b2-25653f0171ee" (UID: "1a24e9f0-98ce-4367-99b2-25653f0171ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:23.202186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.201871 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-workload-socket" pod "1a24e9f0-98ce-4367-99b2-25653f0171ee" (UID: "1a24e9f0-98ce-4367-99b2-25653f0171ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.202186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.202089 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-workload-certs" pod "1a24e9f0-98ce-4367-99b2-25653f0171ee" (UID: "1a24e9f0-98ce-4367-99b2-25653f0171ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.202186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.202153 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-data" pod "1a24e9f0-98ce-4367-99b2-25653f0171ee" (UID: "1a24e9f0-98ce-4367-99b2-25653f0171ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.202839 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.202797 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-credential-socket" pod "1a24e9f0-98ce-4367-99b2-25653f0171ee" (UID: "1a24e9f0-98ce-4367-99b2-25653f0171ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.210619 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.210582 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-token" pod "1a24e9f0-98ce-4367-99b2-25653f0171ee" (UID: "1a24e9f0-98ce-4367-99b2-25653f0171ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:23.210959 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.210823 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a24e9f0-98ce-4367-99b2-25653f0171ee-kube-api-access-plgnk" pod "1a24e9f0-98ce-4367-99b2-25653f0171ee" (UID: "1a24e9f0-98ce-4367-99b2-25653f0171ee"). InnerVolumeSpecName "kube-api-access-plgnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:23.212481 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.212448 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-envoy" pod "1a24e9f0-98ce-4367-99b2-25653f0171ee" (UID: "1a24e9f0-98ce-4367-99b2-25653f0171ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.227758 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.227678 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:35:23.237985 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.237930 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-podinfo" pod "1a24e9f0-98ce-4367-99b2-25653f0171ee" (UID: "1a24e9f0-98ce-4367-99b2-25653f0171ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:35:23.271580 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.270759 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:23.280216 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.280179 5056 generic.go:356] "Generic (PLEG): container finished" podID="1a24e9f0-98ce-4367-99b2-25653f0171ee" containerID="33da25dafe61f4d9fe53fd9107ffad5b16ffdf5ad21448598f3af69d1da459ff" exitCode=0 Oct 29 23:35:23.280402 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.280271 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" event={"ID":"1a24e9f0-98ce-4367-99b2-25653f0171ee","Type":"ContainerDied","Data":"33da25dafe61f4d9fe53fd9107ffad5b16ffdf5ad21448598f3af69d1da459ff"} Oct 29 23:35:23.280402 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.280351 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" event={"ID":"1a24e9f0-98ce-4367-99b2-25653f0171ee","Type":"ContainerDied","Data":"e780e2b42d5f611882fc5b47bd7ed061947048ce4e16dce7fb2e99ae0c92ce62"} Oct 29 23:35:23.280402 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.280383 5056 scope.go:122] "RemoveContainer" containerID="33da25dafe61f4d9fe53fd9107ffad5b16ffdf5ad21448598f3af69d1da459ff" Oct 29 23:35:23.280624 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.280608 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v" Oct 29 23:35:23.288343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.288316 5056 generic.go:356] "Generic (PLEG): container finished" podID="b6e9182f-276e-4ca6-bf97-16a1cf21ba00" containerID="1fa9b3b5e91f9925de87dc7bf346c65a6825f6462ae86ef37ddf1cb9687de5b9" exitCode=0 Oct 29 23:35:23.288467 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.288435 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" event={"ID":"b6e9182f-276e-4ca6-bf97-16a1cf21ba00","Type":"ContainerDied","Data":"1fa9b3b5e91f9925de87dc7bf346c65a6825f6462ae86ef37ddf1cb9687de5b9"} Oct 29 23:35:23.289971 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.289952 5056 generic.go:356] "Generic (PLEG): container finished" podID="7e1dad7e-beb0-49c7-be70-d99766042790" containerID="de2fd2b39bb6ebdc532019edc3e4712feb3c6c93a04f5d79c5748808349dce87" exitCode=0 Oct 29 23:35:23.290074 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.290022 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" event={"ID":"7e1dad7e-beb0-49c7-be70-d99766042790","Type":"ContainerDied","Data":"de2fd2b39bb6ebdc532019edc3e4712feb3c6c93a04f5d79c5748808349dce87"} Oct 29 23:35:23.290074 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.290060 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" event={"ID":"7e1dad7e-beb0-49c7-be70-d99766042790","Type":"ContainerDied","Data":"803daf11eb389d3311c37ca18c91bd9eff0ed7bdce855556c113f5dad1fb6ceb"} Oct 29 23:35:23.290192 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.290131 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt" Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.299317 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-workload-certs\") pod \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.299427 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-kube-api-access-xx85s\" (UniqueName: \"kubernetes.io/projected/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-kube-api-access-xx85s\") pod \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.299461 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-workload-socket\") pod \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.299492 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-data\") pod \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.299591 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-token\" (UniqueName: \"kubernetes.io/projected/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-token\") pod \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.299667 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-podinfo\") pod \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.299699 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-envoy\") pod \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.299751 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-credential-socket\") pod \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.299783 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istiod-ca-cert\") pod \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\" (UID: \"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850\") " Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.300176 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plgnk\" (UniqueName: \"kubernetes.io/projected/1a24e9f0-98ce-4367-99b2-25653f0171ee-kube-api-access-plgnk\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.300197 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.300213 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.300227 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.300242 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a24e9f0-98ce-4367-99b2-25653f0171ee-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.300255 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.300285 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.300296 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a24e9f0-98ce-4367-99b2-25653f0171ee-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.300309 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a24e9f0-98ce-4367-99b2-25653f0171ee-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.300727 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-data" pod "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" (UID: "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.301713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.300909 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-workload-certs" pod "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" (UID: "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.303457 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.302482 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istiod-ca-cert" pod "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" (UID: "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:23.311983 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.311960 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-credential-socket" pod "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" (UID: "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.312208 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.312188 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-workload-socket" pod "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" (UID: "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.324050 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.320921 5056 generic.go:356] "Generic (PLEG): container finished" podID="9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" containerID="9f3d57123fab029cee58404e9328eb28102fb843df4e1b141f42fa2f264f46d3" exitCode=0 Oct 29 23:35:23.324050 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.321060 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" event={"ID":"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850","Type":"ContainerDied","Data":"9f3d57123fab029cee58404e9328eb28102fb843df4e1b141f42fa2f264f46d3"} Oct 29 23:35:23.324050 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.321110 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" event={"ID":"9f88e972-ce97-4ecc-83dc-0b4a2c4bf850","Type":"ContainerDied","Data":"ef51d98cb4eb69fb6468a086c421aa586644535b3d5dbf0ced2ba09f7c1365f6"} Oct 29 23:35:23.324050 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.321189 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc" Oct 29 23:35:23.338817 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.338766 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-envoy" pod "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" (UID: "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.344203 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.342239 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-token" pod "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" (UID: "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:23.344203 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.344157 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-kube-api-access-xx85s" pod "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" (UID: "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850"). InnerVolumeSpecName "kube-api-access-xx85s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:23.349835 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.349794 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-podinfo" pod "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" (UID: "9f88e972-ce97-4ecc-83dc-0b4a2c4bf850"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:35:23.413874 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.413839 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt"] Oct 29 23:35:23.420349 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.420317 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.426342 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.426315 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.427318 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.427295 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.427467 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.427451 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.478381 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.446347 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.478381 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.446403 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.478381 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.446418 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xx85s\" (UniqueName: \"kubernetes.io/projected/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-kube-api-access-xx85s\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.478381 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.446435 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.478381 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.446450 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.478381 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.425490 5056 scope.go:122] "RemoveContainer" containerID="c7c965a4faa698e70a94442ca3e2686c47cb0c57651d2d94f414611b6cf3ca6b" Oct 29 23:35:23.517505 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.517474 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:35:23.722686 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.722608 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qffdt"] Oct 29 23:35:23.745287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.745167 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v"] Oct 29 23:35:23.754561 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.754472 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-workload-certs\") pod \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " Oct 29 23:35:23.755947 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.755912 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-data\") pod \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " Oct 29 23:35:23.756134 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.756118 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-token\" (UniqueName: \"kubernetes.io/projected/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-token\") pod \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " Oct 29 23:35:23.756253 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.756238 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-kube-api-access-9b9lc\" (UniqueName: \"kubernetes.io/projected/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-kube-api-access-9b9lc\") pod \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " Oct 29 23:35:23.756388 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.756372 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-podinfo\") pod \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " Oct 29 23:35:23.756545 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.756529 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-envoy\") pod \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " Oct 29 23:35:23.756670 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.756658 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-workload-socket\") pod \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " Oct 29 23:35:23.756788 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.756775 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-credential-socket\") pod \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " Oct 29 23:35:23.756904 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.756889 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istiod-ca-cert\") pod \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\" (UID: \"b6e9182f-276e-4ca6-bf97-16a1cf21ba00\") " Oct 29 23:35:23.758014 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.757982 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istiod-ca-cert" pod "b6e9182f-276e-4ca6-bf97-16a1cf21ba00" (UID: "b6e9182f-276e-4ca6-bf97-16a1cf21ba00"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:23.758685 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.758664 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-workload-certs" pod "b6e9182f-276e-4ca6-bf97-16a1cf21ba00" (UID: "b6e9182f-276e-4ca6-bf97-16a1cf21ba00"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.759014 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.758996 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-data" pod "b6e9182f-276e-4ca6-bf97-16a1cf21ba00" (UID: "b6e9182f-276e-4ca6-bf97-16a1cf21ba00"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.791903 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.791855 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-workload-socket" pod "b6e9182f-276e-4ca6-bf97-16a1cf21ba00" (UID: "b6e9182f-276e-4ca6-bf97-16a1cf21ba00"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.792390 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.792329 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-credential-socket" pod "b6e9182f-276e-4ca6-bf97-16a1cf21ba00" (UID: "b6e9182f-276e-4ca6-bf97-16a1cf21ba00"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.792860 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.792834 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-podinfo" pod "b6e9182f-276e-4ca6-bf97-16a1cf21ba00" (UID: "b6e9182f-276e-4ca6-bf97-16a1cf21ba00"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:35:23.799002 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.798922 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-btd5v"] Oct 29 23:35:23.802289 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.802238 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-token" pod "b6e9182f-276e-4ca6-bf97-16a1cf21ba00" (UID: "b6e9182f-276e-4ca6-bf97-16a1cf21ba00"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:23.809518 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.809463 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-envoy" pod "b6e9182f-276e-4ca6-bf97-16a1cf21ba00" (UID: "b6e9182f-276e-4ca6-bf97-16a1cf21ba00"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:23.809999 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.809965 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-kube-api-access-9b9lc" pod "b6e9182f-276e-4ca6-bf97-16a1cf21ba00" (UID: "b6e9182f-276e-4ca6-bf97-16a1cf21ba00"). InnerVolumeSpecName "kube-api-access-9b9lc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:23.856586 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.854075 5056 scope.go:122] "RemoveContainer" containerID="33da25dafe61f4d9fe53fd9107ffad5b16ffdf5ad21448598f3af69d1da459ff" Oct 29 23:35:23.856586 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:35:23.854888 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33da25dafe61f4d9fe53fd9107ffad5b16ffdf5ad21448598f3af69d1da459ff\": container with ID starting with 33da25dafe61f4d9fe53fd9107ffad5b16ffdf5ad21448598f3af69d1da459ff not found: ID does not exist" containerID="33da25dafe61f4d9fe53fd9107ffad5b16ffdf5ad21448598f3af69d1da459ff" Oct 29 23:35:23.856586 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.854930 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33da25dafe61f4d9fe53fd9107ffad5b16ffdf5ad21448598f3af69d1da459ff"} err="failed to get container status \"33da25dafe61f4d9fe53fd9107ffad5b16ffdf5ad21448598f3af69d1da459ff\": rpc error: code = NotFound desc = could not find container \"33da25dafe61f4d9fe53fd9107ffad5b16ffdf5ad21448598f3af69d1da459ff\": container with ID starting with 33da25dafe61f4d9fe53fd9107ffad5b16ffdf5ad21448598f3af69d1da459ff not found: ID does not exist" Oct 29 23:35:23.856586 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.854956 5056 scope.go:122] "RemoveContainer" containerID="c7c965a4faa698e70a94442ca3e2686c47cb0c57651d2d94f414611b6cf3ca6b" Oct 29 23:35:23.870737 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:35:23.862091 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7c965a4faa698e70a94442ca3e2686c47cb0c57651d2d94f414611b6cf3ca6b\": container with ID starting with c7c965a4faa698e70a94442ca3e2686c47cb0c57651d2d94f414611b6cf3ca6b not found: ID does not exist" containerID="c7c965a4faa698e70a94442ca3e2686c47cb0c57651d2d94f414611b6cf3ca6b" Oct 29 23:35:23.870737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.862131 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7c965a4faa698e70a94442ca3e2686c47cb0c57651d2d94f414611b6cf3ca6b"} err="failed to get container status \"c7c965a4faa698e70a94442ca3e2686c47cb0c57651d2d94f414611b6cf3ca6b\": rpc error: code = NotFound desc = could not find container \"c7c965a4faa698e70a94442ca3e2686c47cb0c57651d2d94f414611b6cf3ca6b\": container with ID starting with c7c965a4faa698e70a94442ca3e2686c47cb0c57651d2d94f414611b6cf3ca6b not found: ID does not exist" Oct 29 23:35:23.870737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.862181 5056 scope.go:122] "RemoveContainer" containerID="de2fd2b39bb6ebdc532019edc3e4712feb3c6c93a04f5d79c5748808349dce87" Oct 29 23:35:23.870737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.862539 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.870737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.862571 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.870737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.862594 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.870737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.862608 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.870737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.862623 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.870737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.862638 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9b9lc\" (UniqueName: \"kubernetes.io/projected/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-kube-api-access-9b9lc\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.870737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.862654 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.870737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.862668 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.870737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.862681 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6e9182f-276e-4ca6-bf97-16a1cf21ba00-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:23.878155 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:35:23.878127 5056 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f88e972_ce97_4ecc_83dc_0b4a2c4bf850.slice\": RecentStats: unable to find data in memory cache]" Oct 29 23:35:23.899121 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.899090 5056 scope.go:122] "RemoveContainer" containerID="de2fd2b39bb6ebdc532019edc3e4712feb3c6c93a04f5d79c5748808349dce87" Oct 29 23:35:23.899678 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:35:23.899653 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de2fd2b39bb6ebdc532019edc3e4712feb3c6c93a04f5d79c5748808349dce87\": container with ID starting with de2fd2b39bb6ebdc532019edc3e4712feb3c6c93a04f5d79c5748808349dce87 not found: ID does not exist" containerID="de2fd2b39bb6ebdc532019edc3e4712feb3c6c93a04f5d79c5748808349dce87" Oct 29 23:35:23.899834 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.899808 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de2fd2b39bb6ebdc532019edc3e4712feb3c6c93a04f5d79c5748808349dce87"} err="failed to get container status \"de2fd2b39bb6ebdc532019edc3e4712feb3c6c93a04f5d79c5748808349dce87\": rpc error: code = NotFound desc = could not find container \"de2fd2b39bb6ebdc532019edc3e4712feb3c6c93a04f5d79c5748808349dce87\": container with ID starting with de2fd2b39bb6ebdc532019edc3e4712feb3c6c93a04f5d79c5748808349dce87 not found: ID does not exist" Oct 29 23:35:23.899933 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.899913 5056 scope.go:122] "RemoveContainer" containerID="9f3d57123fab029cee58404e9328eb28102fb843df4e1b141f42fa2f264f46d3" Oct 29 23:35:23.901833 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.901816 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc"] Oct 29 23:35:23.908527 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:23.908505 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-xfdvc"] Oct 29 23:35:24.026715 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.026624 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:24.122479 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.097644 5056 scope.go:122] "RemoveContainer" containerID="9f3d57123fab029cee58404e9328eb28102fb843df4e1b141f42fa2f264f46d3" Oct 29 23:35:24.145531 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:35:24.145435 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f3d57123fab029cee58404e9328eb28102fb843df4e1b141f42fa2f264f46d3\": container with ID starting with 9f3d57123fab029cee58404e9328eb28102fb843df4e1b141f42fa2f264f46d3 not found: ID does not exist" containerID="9f3d57123fab029cee58404e9328eb28102fb843df4e1b141f42fa2f264f46d3" Oct 29 23:35:24.145531 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.145479 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f3d57123fab029cee58404e9328eb28102fb843df4e1b141f42fa2f264f46d3"} err="failed to get container status \"9f3d57123fab029cee58404e9328eb28102fb843df4e1b141f42fa2f264f46d3\": rpc error: code = NotFound desc = could not find container \"9f3d57123fab029cee58404e9328eb28102fb843df4e1b141f42fa2f264f46d3\": container with ID starting with 9f3d57123fab029cee58404e9328eb28102fb843df4e1b141f42fa2f264f46d3 not found: ID does not exist" Oct 29 23:35:24.296866 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.296777 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:24.512194 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.512160 5056 generic.go:356] "Generic (PLEG): container finished" podID="e80bef80-0024-4113-b7f5-f43eded6d18e" containerID="50637988903da9eee37952b4da1fbccdae355252ceb4ea02e87434770f457c03" exitCode=0 Oct 29 23:35:24.512535 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.512505 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" event={"ID":"e80bef80-0024-4113-b7f5-f43eded6d18e","Type":"ContainerDied","Data":"50637988903da9eee37952b4da1fbccdae355252ceb4ea02e87434770f457c03"} Oct 29 23:35:24.516059 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.516028 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" event={"ID":"b6e9182f-276e-4ca6-bf97-16a1cf21ba00","Type":"ContainerDied","Data":"1141907abf5962e1a6507409321c61ce78c26dbd7c0b55b977b08304b6311d4c"} Oct 29 23:35:24.516254 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.516229 5056 scope.go:122] "RemoveContainer" containerID="1fa9b3b5e91f9925de87dc7bf346c65a6825f6462ae86ef37ddf1cb9687de5b9" Oct 29 23:35:24.516540 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.516523 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w" Oct 29 23:35:24.519705 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.519665 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:35:24.527278 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.524010 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e80bef80-0024-4113-b7f5-f43eded6d18e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e80bef80-0024-4113-b7f5-f43eded6d18e-openshift-service-ca\") pod \"e80bef80-0024-4113-b7f5-f43eded6d18e\" (UID: \"e80bef80-0024-4113-b7f5-f43eded6d18e\") " Oct 29 23:35:24.550327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.527641 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e80bef80-0024-4113-b7f5-f43eded6d18e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e80bef80-0024-4113-b7f5-f43eded6d18e-maas-api-tls\") pod \"e80bef80-0024-4113-b7f5-f43eded6d18e\" (UID: \"e80bef80-0024-4113-b7f5-f43eded6d18e\") " Oct 29 23:35:24.550563 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.550539 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e80bef80-0024-4113-b7f5-f43eded6d18e-kube-api-access-4jttl\" (UniqueName: \"kubernetes.io/projected/e80bef80-0024-4113-b7f5-f43eded6d18e-kube-api-access-4jttl\") pod \"e80bef80-0024-4113-b7f5-f43eded6d18e\" (UID: \"e80bef80-0024-4113-b7f5-f43eded6d18e\") " Oct 29 23:35:24.550760 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.550743 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e80bef80-0024-4113-b7f5-f43eded6d18e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e80bef80-0024-4113-b7f5-f43eded6d18e-metrics-certs\") pod \"e80bef80-0024-4113-b7f5-f43eded6d18e\" (UID: \"e80bef80-0024-4113-b7f5-f43eded6d18e\") " Oct 29 23:35:24.552311 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.525610 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e80bef80-0024-4113-b7f5-f43eded6d18e-openshift-service-ca" pod "e80bef80-0024-4113-b7f5-f43eded6d18e" (UID: "e80bef80-0024-4113-b7f5-f43eded6d18e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:24.563783 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.563740 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e80bef80-0024-4113-b7f5-f43eded6d18e-kube-api-access-4jttl" pod "e80bef80-0024-4113-b7f5-f43eded6d18e" (UID: "e80bef80-0024-4113-b7f5-f43eded6d18e"). InnerVolumeSpecName "kube-api-access-4jttl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:24.662526 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.662433 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jttl\" (UniqueName: \"kubernetes.io/projected/e80bef80-0024-4113-b7f5-f43eded6d18e-kube-api-access-4jttl\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:24.662526 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.662469 5056 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e80bef80-0024-4113-b7f5-f43eded6d18e-openshift-service-ca\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:24.794306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.781403 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e80bef80-0024-4113-b7f5-f43eded6d18e-maas-api-tls" pod "e80bef80-0024-4113-b7f5-f43eded6d18e" (UID: "e80bef80-0024-4113-b7f5-f43eded6d18e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:35:24.864682 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.862527 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w"] Oct 29 23:35:24.864682 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.864330 5056 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e80bef80-0024-4113-b7f5-f43eded6d18e-maas-api-tls\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:24.890298 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.883423 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e80bef80-0024-4113-b7f5-f43eded6d18e-metrics-certs" pod "e80bef80-0024-4113-b7f5-f43eded6d18e" (UID: "e80bef80-0024-4113-b7f5-f43eded6d18e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:35:24.937290 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.931711 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a24e9f0-98ce-4367-99b2-25653f0171ee" path="/var/lib/kubelet/pods/1a24e9f0-98ce-4367-99b2-25653f0171ee/volumes" Oct 29 23:35:24.938252 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.938228 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e1dad7e-beb0-49c7-be70-d99766042790" path="/var/lib/kubelet/pods/7e1dad7e-beb0-49c7-be70-d99766042790/volumes" Oct 29 23:35:24.948408 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.948386 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:24.948939 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.948905 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" path="/var/lib/kubelet/pods/9f88e972-ce97-4ecc-83dc-0b4a2c4bf850/volumes" Oct 29 23:35:24.961675 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.961653 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-ae1364-openshift-default-5df5f95d69-qls5w"] Oct 29 23:35:24.965474 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:24.965454 5056 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e80bef80-0024-4113-b7f5-f43eded6d18e-metrics-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:25.549393 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:25.529067 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:25.607340 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:25.607300 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" event={"ID":"e80bef80-0024-4113-b7f5-f43eded6d18e","Type":"ContainerDied","Data":"416cc7d2858b1be7fd5faf4f707d7568b8cb4150c8d2cf660a6281ac92c1a97f"} Oct 29 23:35:25.607507 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:25.607373 5056 scope.go:122] "RemoveContainer" containerID="50637988903da9eee37952b4da1fbccdae355252ceb4ea02e87434770f457c03" Oct 29 23:35:25.607507 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:25.607492 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql" Oct 29 23:35:25.693675 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:25.693630 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql"] Oct 29 23:35:25.698471 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:25.698424 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:25.704740 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:25.704709 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-ae1784-558c669c6-gz8ql"] Oct 29 23:35:26.777652 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:26.777623 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b6e9182f-276e-4ca6-bf97-16a1cf21ba00" path="/var/lib/kubelet/pods/b6e9182f-276e-4ca6-bf97-16a1cf21ba00/volumes" Oct 29 23:35:26.778179 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:26.778163 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e80bef80-0024-4113-b7f5-f43eded6d18e" path="/var/lib/kubelet/pods/e80bef80-0024-4113-b7f5-f43eded6d18e/volumes" Oct 29 23:35:28.061438 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:28.061405 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:31.937466 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:31.937428 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:32.167387 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:32.167350 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:32.290790 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:32.290694 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww"] Oct 29 23:35:32.290950 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:32.290917 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" podUID="8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" containerName="istio-proxy" containerID="cri-o://ddab0c9e433ee4cdfacd7c08a145602dbf3b9b903a26295b0ff9a75a79cb38fe" gracePeriod=30 Oct 29 23:35:32.306250 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:32.305442 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g"] Oct 29 23:35:32.306250 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:32.305700 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" podUID="b46bbd7f-4b97-4b30-b500-1c31f0507898" containerName="istio-proxy" containerID="cri-o://750d0b2fd32ea5d6305897f9a7d15d7f239257a14124d662e98bdb381a47fbf4" gracePeriod=30 Oct 29 23:35:32.311327 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:32.311300 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8"] Oct 29 23:35:32.311790 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:32.311743 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" podUID="6d812120-3b8e-46c4-88d7-febeea4f0670" containerName="istio-proxy" containerID="cri-o://75b6ba10c4fdca6c2c65f261457bd420d5bb6d07574ce9e282230d799c581c02" gracePeriod=30 Oct 29 23:35:32.320742 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:32.320716 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z"] Oct 29 23:35:32.321038 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:32.320992 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" podUID="e4050977-832e-4b94-9161-5a23e0e27721" containerName="istio-proxy" containerID="cri-o://713c06808d7c6c455b4eccece2a03c38ea39d7cbb3f70835cf8270092baf4c2d" gracePeriod=30 Oct 29 23:35:32.366484 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:32.366450 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:32.799630 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:32.799594 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:35:32.799792 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:32.799673 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" podUID="e4050977-832e-4b94-9161-5a23e0e27721" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:35:33.462562 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:33.462528 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:35:33.462954 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:33.462590 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" podUID="6d812120-3b8e-46c4-88d7-febeea4f0670" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:35:34.844460 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:34.844423 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:35.019323 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:35.017814 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:35.260288 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:35.260221 5056 patch_prober.go:29] interesting pod/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:35:35.260470 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:35.260320 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" podUID="8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:35:36.508692 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:36.508652 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:36.654286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:36.654224 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk"] Oct 29 23:35:36.654548 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:36.654466 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" podUID="76b760f1-81f7-460a-bb76-1319cd789c50" containerName="maas-api" containerID="cri-o://f8b7cfe3f9e17d2bce35afa6cffc12937971b8dceaefabf115bee36bc9fad204" gracePeriod=30 Oct 29 23:35:36.779784 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:36.779748 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:37.013701 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.013669 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:35:37.102068 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.101971 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/76b760f1-81f7-460a-bb76-1319cd789c50-metrics-certs\" (UniqueName: \"kubernetes.io/secret/76b760f1-81f7-460a-bb76-1319cd789c50-metrics-certs\") pod \"76b760f1-81f7-460a-bb76-1319cd789c50\" (UID: \"76b760f1-81f7-460a-bb76-1319cd789c50\") " Oct 29 23:35:37.102241 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.102087 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76b760f1-81f7-460a-bb76-1319cd789c50-kube-api-access-j2jzz\" (UniqueName: \"kubernetes.io/projected/76b760f1-81f7-460a-bb76-1319cd789c50-kube-api-access-j2jzz\") pod \"76b760f1-81f7-460a-bb76-1319cd789c50\" (UID: \"76b760f1-81f7-460a-bb76-1319cd789c50\") " Oct 29 23:35:37.102241 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.102202 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76b760f1-81f7-460a-bb76-1319cd789c50-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76b760f1-81f7-460a-bb76-1319cd789c50-openshift-service-ca\") pod \"76b760f1-81f7-460a-bb76-1319cd789c50\" (UID: \"76b760f1-81f7-460a-bb76-1319cd789c50\") " Oct 29 23:35:37.102241 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.102232 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/76b760f1-81f7-460a-bb76-1319cd789c50-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76b760f1-81f7-460a-bb76-1319cd789c50-maas-api-tls\") pod \"76b760f1-81f7-460a-bb76-1319cd789c50\" (UID: \"76b760f1-81f7-460a-bb76-1319cd789c50\") " Oct 29 23:35:37.102816 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.102673 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76b760f1-81f7-460a-bb76-1319cd789c50-openshift-service-ca" pod "76b760f1-81f7-460a-bb76-1319cd789c50" (UID: "76b760f1-81f7-460a-bb76-1319cd789c50"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:37.102952 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.102817 5056 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/76b760f1-81f7-460a-bb76-1319cd789c50-openshift-service-ca\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.104671 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.104640 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76b760f1-81f7-460a-bb76-1319cd789c50-metrics-certs" pod "76b760f1-81f7-460a-bb76-1319cd789c50" (UID: "76b760f1-81f7-460a-bb76-1319cd789c50"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:35:37.104791 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.104664 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/76b760f1-81f7-460a-bb76-1319cd789c50-maas-api-tls" pod "76b760f1-81f7-460a-bb76-1319cd789c50" (UID: "76b760f1-81f7-460a-bb76-1319cd789c50"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:35:37.104791 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.104757 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76b760f1-81f7-460a-bb76-1319cd789c50-kube-api-access-j2jzz" pod "76b760f1-81f7-460a-bb76-1319cd789c50" (UID: "76b760f1-81f7-460a-bb76-1319cd789c50"). InnerVolumeSpecName "kube-api-access-j2jzz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:37.216300 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.214759 5056 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/76b760f1-81f7-460a-bb76-1319cd789c50-metrics-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.216300 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.214797 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2jzz\" (UniqueName: \"kubernetes.io/projected/76b760f1-81f7-460a-bb76-1319cd789c50-kube-api-access-j2jzz\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.216300 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.214812 5056 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/76b760f1-81f7-460a-bb76-1319cd789c50-maas-api-tls\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.329960 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.329922 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:37.506285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.501760 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:37.652480 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.652129 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:35:37.719872 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.719841 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:35:37.725486 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.725287 5056 generic.go:356] "Generic (PLEG): container finished" podID="76b760f1-81f7-460a-bb76-1319cd789c50" containerID="f8b7cfe3f9e17d2bce35afa6cffc12937971b8dceaefabf115bee36bc9fad204" exitCode=0 Oct 29 23:35:37.725486 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.725399 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" event={"ID":"76b760f1-81f7-460a-bb76-1319cd789c50","Type":"ContainerDied","Data":"f8b7cfe3f9e17d2bce35afa6cffc12937971b8dceaefabf115bee36bc9fad204"} Oct 29 23:35:37.725648 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.725486 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" event={"ID":"76b760f1-81f7-460a-bb76-1319cd789c50","Type":"ContainerDied","Data":"d90e59ece49b2901dfe9165545948412a34d4168cced2b578f42f73a33685e97"} Oct 29 23:35:37.725648 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.725517 5056 scope.go:122] "RemoveContainer" containerID="f8b7cfe3f9e17d2bce35afa6cffc12937971b8dceaefabf115bee36bc9fad204" Oct 29 23:35:37.725742 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.725661 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk" Oct 29 23:35:37.736156 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.736127 5056 generic.go:356] "Generic (PLEG): container finished" podID="b46bbd7f-4b97-4b30-b500-1c31f0507898" containerID="750d0b2fd32ea5d6305897f9a7d15d7f239257a14124d662e98bdb381a47fbf4" exitCode=0 Oct 29 23:35:37.736629 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.736606 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" event={"ID":"b46bbd7f-4b97-4b30-b500-1c31f0507898","Type":"ContainerDied","Data":"750d0b2fd32ea5d6305897f9a7d15d7f239257a14124d662e98bdb381a47fbf4"} Oct 29 23:35:37.736840 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.736812 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" event={"ID":"b46bbd7f-4b97-4b30-b500-1c31f0507898","Type":"ContainerDied","Data":"70339fbf635eff23c76a09d889f1220de8fc440393a85c59404ef06a58ce0e15"} Oct 29 23:35:37.736979 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.736956 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g" Oct 29 23:35:37.744448 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.744425 5056 generic.go:356] "Generic (PLEG): container finished" podID="e4050977-832e-4b94-9161-5a23e0e27721" containerID="713c06808d7c6c455b4eccece2a03c38ea39d7cbb3f70835cf8270092baf4c2d" exitCode=0 Oct 29 23:35:37.744559 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.744458 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" event={"ID":"e4050977-832e-4b94-9161-5a23e0e27721","Type":"ContainerDied","Data":"713c06808d7c6c455b4eccece2a03c38ea39d7cbb3f70835cf8270092baf4c2d"} Oct 29 23:35:37.752186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.746721 5056 generic.go:356] "Generic (PLEG): container finished" podID="8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" containerID="ddab0c9e433ee4cdfacd7c08a145602dbf3b9b903a26295b0ff9a75a79cb38fe" exitCode=0 Oct 29 23:35:37.752186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.746794 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" event={"ID":"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a","Type":"ContainerDied","Data":"ddab0c9e433ee4cdfacd7c08a145602dbf3b9b903a26295b0ff9a75a79cb38fe"} Oct 29 23:35:37.752186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.750834 5056 generic.go:356] "Generic (PLEG): container finished" podID="6d812120-3b8e-46c4-88d7-febeea4f0670" containerID="75b6ba10c4fdca6c2c65f261457bd420d5bb6d07574ce9e282230d799c581c02" exitCode=0 Oct 29 23:35:37.752186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.750867 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" event={"ID":"6d812120-3b8e-46c4-88d7-febeea4f0670","Type":"ContainerDied","Data":"75b6ba10c4fdca6c2c65f261457bd420d5bb6d07574ce9e282230d799c581c02"} Oct 29 23:35:37.752186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.750904 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" event={"ID":"6d812120-3b8e-46c4-88d7-febeea4f0670","Type":"ContainerDied","Data":"ae9e61cae56ecfdc115e7eff3e934c53b5cee37bf1deba4a74ab21b17ed71ea2"} Oct 29 23:35:37.752186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.750980 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8" Oct 29 23:35:37.777673 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.771426 5056 scope.go:122] "RemoveContainer" containerID="f8b7cfe3f9e17d2bce35afa6cffc12937971b8dceaefabf115bee36bc9fad204" Oct 29 23:35:37.777673 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:35:37.773667 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8b7cfe3f9e17d2bce35afa6cffc12937971b8dceaefabf115bee36bc9fad204\": container with ID starting with f8b7cfe3f9e17d2bce35afa6cffc12937971b8dceaefabf115bee36bc9fad204 not found: ID does not exist" containerID="f8b7cfe3f9e17d2bce35afa6cffc12937971b8dceaefabf115bee36bc9fad204" Oct 29 23:35:37.777673 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.773697 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8b7cfe3f9e17d2bce35afa6cffc12937971b8dceaefabf115bee36bc9fad204"} err="failed to get container status \"f8b7cfe3f9e17d2bce35afa6cffc12937971b8dceaefabf115bee36bc9fad204\": rpc error: code = NotFound desc = could not find container \"f8b7cfe3f9e17d2bce35afa6cffc12937971b8dceaefabf115bee36bc9fad204\": container with ID starting with f8b7cfe3f9e17d2bce35afa6cffc12937971b8dceaefabf115bee36bc9fad204 not found: ID does not exist" Oct 29 23:35:37.777673 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.773717 5056 scope.go:122] "RemoveContainer" containerID="750d0b2fd32ea5d6305897f9a7d15d7f239257a14124d662e98bdb381a47fbf4" Oct 29 23:35:37.788413 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.788337 5056 scope.go:122] "RemoveContainer" containerID="b684f331163d46ae3a985263979830c8fd0d38f46e2c5df184da198dd480426d" Oct 29 23:35:37.792115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.792091 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:35:37.800230 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.800204 5056 scope.go:122] "RemoveContainer" containerID="750d0b2fd32ea5d6305897f9a7d15d7f239257a14124d662e98bdb381a47fbf4" Oct 29 23:35:37.800937 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:35:37.800725 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"750d0b2fd32ea5d6305897f9a7d15d7f239257a14124d662e98bdb381a47fbf4\": container with ID starting with 750d0b2fd32ea5d6305897f9a7d15d7f239257a14124d662e98bdb381a47fbf4 not found: ID does not exist" containerID="750d0b2fd32ea5d6305897f9a7d15d7f239257a14124d662e98bdb381a47fbf4" Oct 29 23:35:37.800937 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.800761 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"750d0b2fd32ea5d6305897f9a7d15d7f239257a14124d662e98bdb381a47fbf4"} err="failed to get container status \"750d0b2fd32ea5d6305897f9a7d15d7f239257a14124d662e98bdb381a47fbf4\": rpc error: code = NotFound desc = could not find container \"750d0b2fd32ea5d6305897f9a7d15d7f239257a14124d662e98bdb381a47fbf4\": container with ID starting with 750d0b2fd32ea5d6305897f9a7d15d7f239257a14124d662e98bdb381a47fbf4 not found: ID does not exist" Oct 29 23:35:37.800937 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.800785 5056 scope.go:122] "RemoveContainer" containerID="b684f331163d46ae3a985263979830c8fd0d38f46e2c5df184da198dd480426d" Oct 29 23:35:37.801153 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:35:37.801084 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b684f331163d46ae3a985263979830c8fd0d38f46e2c5df184da198dd480426d\": container with ID starting with b684f331163d46ae3a985263979830c8fd0d38f46e2c5df184da198dd480426d not found: ID does not exist" containerID="b684f331163d46ae3a985263979830c8fd0d38f46e2c5df184da198dd480426d" Oct 29 23:35:37.801153 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.801101 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b684f331163d46ae3a985263979830c8fd0d38f46e2c5df184da198dd480426d"} err="failed to get container status \"b684f331163d46ae3a985263979830c8fd0d38f46e2c5df184da198dd480426d\": rpc error: code = NotFound desc = could not find container \"b684f331163d46ae3a985263979830c8fd0d38f46e2c5df184da198dd480426d\": container with ID starting with b684f331163d46ae3a985263979830c8fd0d38f46e2c5df184da198dd480426d not found: ID does not exist" Oct 29 23:35:37.801153 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.801114 5056 scope.go:122] "RemoveContainer" containerID="75b6ba10c4fdca6c2c65f261457bd420d5bb6d07574ce9e282230d799c581c02" Oct 29 23:35:37.824583 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.824552 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-workload-certs\") pod \"6d812120-3b8e-46c4-88d7-febeea4f0670\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " Oct 29 23:35:37.824717 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.824586 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b46bbd7f-4b97-4b30-b500-1c31f0507898-kube-api-access-q7skr\" (UniqueName: \"kubernetes.io/projected/b46bbd7f-4b97-4b30-b500-1c31f0507898-kube-api-access-q7skr\") pod \"b46bbd7f-4b97-4b30-b500-1c31f0507898\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " Oct 29 23:35:37.824717 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.824636 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-credential-socket\") pod \"6d812120-3b8e-46c4-88d7-febeea4f0670\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " Oct 29 23:35:37.824717 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.824687 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d812120-3b8e-46c4-88d7-febeea4f0670-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d812120-3b8e-46c4-88d7-febeea4f0670-istiod-ca-cert\") pod \"6d812120-3b8e-46c4-88d7-febeea4f0670\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " Oct 29 23:35:37.824717 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.824715 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-token\" (UniqueName: \"kubernetes.io/projected/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-token\") pod \"6d812120-3b8e-46c4-88d7-febeea4f0670\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " Oct 29 23:35:37.824938 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.824759 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-workload-socket\") pod \"6d812120-3b8e-46c4-88d7-febeea4f0670\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " Oct 29 23:35:37.824938 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.824821 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-data\") pod \"6d812120-3b8e-46c4-88d7-febeea4f0670\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " Oct 29 23:35:37.824938 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.824898 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-envoy\") pod \"6d812120-3b8e-46c4-88d7-febeea4f0670\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " Oct 29 23:35:37.824938 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.824932 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-podinfo\") pod \"6d812120-3b8e-46c4-88d7-febeea4f0670\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " Oct 29 23:35:37.825138 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.824973 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-credential-socket\") pod \"b46bbd7f-4b97-4b30-b500-1c31f0507898\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " Oct 29 23:35:37.825138 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.825002 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-data\") pod \"b46bbd7f-4b97-4b30-b500-1c31f0507898\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " Oct 29 23:35:37.825138 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.825052 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b46bbd7f-4b97-4b30-b500-1c31f0507898-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b46bbd7f-4b97-4b30-b500-1c31f0507898-istiod-ca-cert\") pod \"b46bbd7f-4b97-4b30-b500-1c31f0507898\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " Oct 29 23:35:37.825138 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.825106 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d812120-3b8e-46c4-88d7-febeea4f0670-kube-api-access-dlmr7\" (UniqueName: \"kubernetes.io/projected/6d812120-3b8e-46c4-88d7-febeea4f0670-kube-api-access-dlmr7\") pod \"6d812120-3b8e-46c4-88d7-febeea4f0670\" (UID: \"6d812120-3b8e-46c4-88d7-febeea4f0670\") " Oct 29 23:35:37.825424 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.825234 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-envoy\") pod \"b46bbd7f-4b97-4b30-b500-1c31f0507898\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " Oct 29 23:35:37.825424 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.825289 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-token\" (UniqueName: \"kubernetes.io/projected/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-token\") pod \"b46bbd7f-4b97-4b30-b500-1c31f0507898\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " Oct 29 23:35:37.825424 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.825325 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-workload-certs\") pod \"b46bbd7f-4b97-4b30-b500-1c31f0507898\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " Oct 29 23:35:37.825424 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.825358 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-podinfo\") pod \"b46bbd7f-4b97-4b30-b500-1c31f0507898\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " Oct 29 23:35:37.825424 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.825389 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-workload-socket\") pod \"b46bbd7f-4b97-4b30-b500-1c31f0507898\" (UID: \"b46bbd7f-4b97-4b30-b500-1c31f0507898\") " Oct 29 23:35:37.825684 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.825431 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-data" pod "6d812120-3b8e-46c4-88d7-febeea4f0670" (UID: "6d812120-3b8e-46c4-88d7-febeea4f0670"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:37.825782 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.825757 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d812120-3b8e-46c4-88d7-febeea4f0670-istiod-ca-cert" pod "6d812120-3b8e-46c4-88d7-febeea4f0670" (UID: "6d812120-3b8e-46c4-88d7-febeea4f0670"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:37.825920 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.825896 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-credential-socket" pod "6d812120-3b8e-46c4-88d7-febeea4f0670" (UID: "6d812120-3b8e-46c4-88d7-febeea4f0670"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:37.826006 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.825987 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-workload-socket" pod "b46bbd7f-4b97-4b30-b500-1c31f0507898" (UID: "b46bbd7f-4b97-4b30-b500-1c31f0507898"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:37.826304 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.826113 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.826304 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.826136 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d812120-3b8e-46c4-88d7-febeea4f0670-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.826719 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.826507 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-workload-socket" pod "6d812120-3b8e-46c4-88d7-febeea4f0670" (UID: "6d812120-3b8e-46c4-88d7-febeea4f0670"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:37.826719 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.826540 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-credential-socket" pod "b46bbd7f-4b97-4b30-b500-1c31f0507898" (UID: "b46bbd7f-4b97-4b30-b500-1c31f0507898"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:37.826719 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.826556 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-data" pod "b46bbd7f-4b97-4b30-b500-1c31f0507898" (UID: "b46bbd7f-4b97-4b30-b500-1c31f0507898"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:37.826916 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.826818 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b46bbd7f-4b97-4b30-b500-1c31f0507898-istiod-ca-cert" pod "b46bbd7f-4b97-4b30-b500-1c31f0507898" (UID: "b46bbd7f-4b97-4b30-b500-1c31f0507898"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:37.827319 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.827043 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-workload-certs" pod "b46bbd7f-4b97-4b30-b500-1c31f0507898" (UID: "b46bbd7f-4b97-4b30-b500-1c31f0507898"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:37.827534 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.827498 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-workload-certs" pod "6d812120-3b8e-46c4-88d7-febeea4f0670" (UID: "6d812120-3b8e-46c4-88d7-febeea4f0670"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:37.832572 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.832551 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk"] Oct 29 23:35:37.836282 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.836203 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-envoy" pod "b46bbd7f-4b97-4b30-b500-1c31f0507898" (UID: "b46bbd7f-4b97-4b30-b500-1c31f0507898"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:37.836282 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.836291 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-aa7e96-56fd4b7ff7-ksktk"] Oct 29 23:35:37.836475 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.836371 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-podinfo" pod "6d812120-3b8e-46c4-88d7-febeea4f0670" (UID: "6d812120-3b8e-46c4-88d7-febeea4f0670"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:35:37.836693 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.836676 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-token" pod "6d812120-3b8e-46c4-88d7-febeea4f0670" (UID: "6d812120-3b8e-46c4-88d7-febeea4f0670"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:37.836826 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.836789 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d812120-3b8e-46c4-88d7-febeea4f0670-kube-api-access-dlmr7" pod "6d812120-3b8e-46c4-88d7-febeea4f0670" (UID: "6d812120-3b8e-46c4-88d7-febeea4f0670"). InnerVolumeSpecName "kube-api-access-dlmr7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:37.837737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.837716 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-envoy" pod "6d812120-3b8e-46c4-88d7-febeea4f0670" (UID: "6d812120-3b8e-46c4-88d7-febeea4f0670"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:37.838058 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.838038 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-podinfo" pod "b46bbd7f-4b97-4b30-b500-1c31f0507898" (UID: "b46bbd7f-4b97-4b30-b500-1c31f0507898"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:35:37.839244 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.839218 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-token" pod "b46bbd7f-4b97-4b30-b500-1c31f0507898" (UID: "b46bbd7f-4b97-4b30-b500-1c31f0507898"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:37.839628 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.839610 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b46bbd7f-4b97-4b30-b500-1c31f0507898-kube-api-access-q7skr" pod "b46bbd7f-4b97-4b30-b500-1c31f0507898" (UID: "b46bbd7f-4b97-4b30-b500-1c31f0507898"). InnerVolumeSpecName "kube-api-access-q7skr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:37.852699 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.852678 5056 scope.go:122] "RemoveContainer" containerID="75b6ba10c4fdca6c2c65f261457bd420d5bb6d07574ce9e282230d799c581c02" Oct 29 23:35:37.853051 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:35:37.853022 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75b6ba10c4fdca6c2c65f261457bd420d5bb6d07574ce9e282230d799c581c02\": container with ID starting with 75b6ba10c4fdca6c2c65f261457bd420d5bb6d07574ce9e282230d799c581c02 not found: ID does not exist" containerID="75b6ba10c4fdca6c2c65f261457bd420d5bb6d07574ce9e282230d799c581c02" Oct 29 23:35:37.853154 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.853059 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75b6ba10c4fdca6c2c65f261457bd420d5bb6d07574ce9e282230d799c581c02"} err="failed to get container status \"75b6ba10c4fdca6c2c65f261457bd420d5bb6d07574ce9e282230d799c581c02\": rpc error: code = NotFound desc = could not find container \"75b6ba10c4fdca6c2c65f261457bd420d5bb6d07574ce9e282230d799c581c02\": container with ID starting with 75b6ba10c4fdca6c2c65f261457bd420d5bb6d07574ce9e282230d799c581c02 not found: ID does not exist" Oct 29 23:35:37.857076 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.857057 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:35:37.927441 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.927356 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-workload-certs\") pod \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " Oct 29 23:35:37.927441 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.927404 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-kube-api-access-thjn7\" (UniqueName: \"kubernetes.io/projected/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-kube-api-access-thjn7\") pod \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " Oct 29 23:35:37.927669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.927440 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-podinfo\") pod \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " Oct 29 23:35:37.927669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.927492 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-token\" (UniqueName: \"kubernetes.io/projected/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-token\") pod \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " Oct 29 23:35:37.927669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.927551 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istiod-ca-cert\") pod \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " Oct 29 23:35:37.927851 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.927680 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-credential-socket\") pod \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " Oct 29 23:35:37.927851 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.927689 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-workload-certs" pod "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" (UID: "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:37.927851 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.927756 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-workload-socket\") pod \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " Oct 29 23:35:37.927851 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.927799 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-envoy\") pod \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " Oct 29 23:35:37.927851 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.927836 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-data\") pod \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\" (UID: \"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a\") " Oct 29 23:35:37.928221 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928194 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-credential-socket" pod "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" (UID: "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:37.928338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928246 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.928338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928283 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.928338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928299 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.928338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928312 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.928338 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928325 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.928594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928338 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b46bbd7f-4b97-4b30-b500-1c31f0507898-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.928594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928341 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istiod-ca-cert" pod "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" (UID: "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:37.928594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928351 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlmr7\" (UniqueName: \"kubernetes.io/projected/6d812120-3b8e-46c4-88d7-febeea4f0670-kube-api-access-dlmr7\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.928594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928364 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.928594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928378 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.928594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928390 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.928594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928403 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b46bbd7f-4b97-4b30-b500-1c31f0507898-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.928594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928416 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b46bbd7f-4b97-4b30-b500-1c31f0507898-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.928594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928431 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.928594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928444 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.928594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928457 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7skr\" (UniqueName: \"kubernetes.io/projected/b46bbd7f-4b97-4b30-b500-1c31f0507898-kube-api-access-q7skr\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.928594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928470 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d812120-3b8e-46c4-88d7-febeea4f0670-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.928594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928484 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d812120-3b8e-46c4-88d7-febeea4f0670-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:37.929238 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928726 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-data" pod "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" (UID: "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:37.929238 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.928888 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-workload-socket" pod "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" (UID: "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:37.930864 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.930833 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-podinfo" pod "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" (UID: "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:35:37.931577 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.931552 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-kube-api-access-thjn7" pod "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" (UID: "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a"). InnerVolumeSpecName "kube-api-access-thjn7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:37.932168 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.932138 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-token" pod "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" (UID: "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:37.932845 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.932825 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-envoy" pod "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" (UID: "8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:37.952004 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:37.951908 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:38.029351 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.029309 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4050977-832e-4b94-9161-5a23e0e27721-kube-api-access-rzwvl\" (UniqueName: \"kubernetes.io/projected/e4050977-832e-4b94-9161-5a23e0e27721-kube-api-access-rzwvl\") pod \"e4050977-832e-4b94-9161-5a23e0e27721\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " Oct 29 23:35:38.029529 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.029387 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-istio-data\") pod \"e4050977-832e-4b94-9161-5a23e0e27721\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " Oct 29 23:35:38.029529 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.029438 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-workload-socket\") pod \"e4050977-832e-4b94-9161-5a23e0e27721\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " Oct 29 23:35:38.029529 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.029476 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-workload-certs\") pod \"e4050977-832e-4b94-9161-5a23e0e27721\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " Oct 29 23:35:38.029529 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.029500 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-istio-envoy\") pod \"e4050977-832e-4b94-9161-5a23e0e27721\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " Oct 29 23:35:38.029760 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.029572 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4050977-832e-4b94-9161-5a23e0e27721-istio-token\" (UniqueName: \"kubernetes.io/projected/e4050977-832e-4b94-9161-5a23e0e27721-istio-token\") pod \"e4050977-832e-4b94-9161-5a23e0e27721\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " Oct 29 23:35:38.029760 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.029635 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e4050977-832e-4b94-9161-5a23e0e27721-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4050977-832e-4b94-9161-5a23e0e27721-istio-podinfo\") pod \"e4050977-832e-4b94-9161-5a23e0e27721\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " Oct 29 23:35:38.029760 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.029671 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e4050977-832e-4b94-9161-5a23e0e27721-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4050977-832e-4b94-9161-5a23e0e27721-istiod-ca-cert\") pod \"e4050977-832e-4b94-9161-5a23e0e27721\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " Oct 29 23:35:38.029760 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.029688 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-istio-data" pod "e4050977-832e-4b94-9161-5a23e0e27721" (UID: "e4050977-832e-4b94-9161-5a23e0e27721"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:38.029760 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.029715 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-credential-socket\") pod \"e4050977-832e-4b94-9161-5a23e0e27721\" (UID: \"e4050977-832e-4b94-9161-5a23e0e27721\") " Oct 29 23:35:38.030195 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.029779 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-workload-socket" pod "e4050977-832e-4b94-9161-5a23e0e27721" (UID: "e4050977-832e-4b94-9161-5a23e0e27721"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:38.030195 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.029794 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-workload-certs" pod "e4050977-832e-4b94-9161-5a23e0e27721" (UID: "e4050977-832e-4b94-9161-5a23e0e27721"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:38.030195 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.029926 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-credential-socket" pod "e4050977-832e-4b94-9161-5a23e0e27721" (UID: "e4050977-832e-4b94-9161-5a23e0e27721"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:38.030195 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.030117 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4050977-832e-4b94-9161-5a23e0e27721-istiod-ca-cert" pod "e4050977-832e-4b94-9161-5a23e0e27721" (UID: "e4050977-832e-4b94-9161-5a23e0e27721"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:38.031121 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.031091 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4050977-832e-4b94-9161-5a23e0e27721-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:38.031121 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.031121 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:38.031343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.031137 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:38.031343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.031152 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:38.031343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.031166 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:38.031343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.031179 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:38.031343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.031191 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:38.031343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.031203 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:38.031343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.031216 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thjn7\" (UniqueName: \"kubernetes.io/projected/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-kube-api-access-thjn7\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:38.031343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.031230 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:38.031343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.031243 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:38.031343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.031255 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:38.031343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.031283 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:38.032618 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.032464 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-istio-envoy" pod "e4050977-832e-4b94-9161-5a23e0e27721" (UID: "e4050977-832e-4b94-9161-5a23e0e27721"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:38.032618 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.032604 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4050977-832e-4b94-9161-5a23e0e27721-istio-token" pod "e4050977-832e-4b94-9161-5a23e0e27721" (UID: "e4050977-832e-4b94-9161-5a23e0e27721"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:38.032781 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.032628 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4050977-832e-4b94-9161-5a23e0e27721-kube-api-access-rzwvl" pod "e4050977-832e-4b94-9161-5a23e0e27721" (UID: "e4050977-832e-4b94-9161-5a23e0e27721"). InnerVolumeSpecName "kube-api-access-rzwvl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:38.032781 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.032645 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4050977-832e-4b94-9161-5a23e0e27721-istio-podinfo" pod "e4050977-832e-4b94-9161-5a23e0e27721" (UID: "e4050977-832e-4b94-9161-5a23e0e27721"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:35:38.089091 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.089060 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g"] Oct 29 23:35:38.095710 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.095683 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-42g4g"] Oct 29 23:35:38.132133 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.132090 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzwvl\" (UniqueName: \"kubernetes.io/projected/e4050977-832e-4b94-9161-5a23e0e27721-kube-api-access-rzwvl\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:38.132133 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.132126 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4050977-832e-4b94-9161-5a23e0e27721-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:38.132133 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.132139 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4050977-832e-4b94-9161-5a23e0e27721-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:38.132427 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.132151 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4050977-832e-4b94-9161-5a23e0e27721-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:38.134325 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.134253 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8"] Oct 29 23:35:38.137849 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.137826 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-nngc8"] Oct 29 23:35:38.757983 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.757958 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" Oct 29 23:35:38.758414 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.757952 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww" event={"ID":"8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a","Type":"ContainerDied","Data":"31e44d2b447c8927d4c0a0167327fff0b03b16bf1dd103ae35c3fe09b9220148"} Oct 29 23:35:38.758414 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.758081 5056 scope.go:122] "RemoveContainer" containerID="ddab0c9e433ee4cdfacd7c08a145602dbf3b9b903a26295b0ff9a75a79cb38fe" Oct 29 23:35:38.762613 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.762595 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" Oct 29 23:35:38.773513 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.771724 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d812120-3b8e-46c4-88d7-febeea4f0670" path="/var/lib/kubelet/pods/6d812120-3b8e-46c4-88d7-febeea4f0670/volumes" Oct 29 23:35:38.773513 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.772285 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76b760f1-81f7-460a-bb76-1319cd789c50" path="/var/lib/kubelet/pods/76b760f1-81f7-460a-bb76-1319cd789c50/volumes" Oct 29 23:35:38.773513 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.772772 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b46bbd7f-4b97-4b30-b500-1c31f0507898" path="/var/lib/kubelet/pods/b46bbd7f-4b97-4b30-b500-1c31f0507898/volumes" Oct 29 23:35:38.773829 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.773798 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z" event={"ID":"e4050977-832e-4b94-9161-5a23e0e27721","Type":"ContainerDied","Data":"7dfa8b2ef3605803a20642d2a734fa325e540a7464aab3cdbcfb4460e53bdd56"} Oct 29 23:35:38.773913 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.773844 5056 scope.go:122] "RemoveContainer" containerID="713c06808d7c6c455b4eccece2a03c38ea39d7cbb3f70835cf8270092baf4c2d" Oct 29 23:35:38.814844 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.814817 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww"] Oct 29 23:35:38.826364 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.826339 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-ht9ww"] Oct 29 23:35:38.866044 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.866002 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z"] Oct 29 23:35:38.874515 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:38.874491 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-ae1784-openshift-default-5b66bf84bf-lq67z"] Oct 29 23:35:40.770069 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:40.770038 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" path="/var/lib/kubelet/pods/8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a/volumes" Oct 29 23:35:40.770505 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:40.770489 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e4050977-832e-4b94-9161-5a23e0e27721" path="/var/lib/kubelet/pods/e4050977-832e-4b94-9161-5a23e0e27721/volumes" Oct 29 23:35:42.565862 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:42.565829 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:44.164754 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:44.164715 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:44.361045 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:44.361004 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:44.481428 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:44.480411 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z"] Oct 29 23:35:44.481428 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:44.480708 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" podUID="80bda72d-4e80-497e-befe-514dcaff1e39" containerName="istio-proxy" containerID="cri-o://70ae58b3d3c9c312981eac19f538081c82c8fa9e43e6382a870fd8a69a040c05" gracePeriod=30 Oct 29 23:35:44.482527 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:44.482490 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn"] Oct 29 23:35:44.482701 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:44.482677 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" podUID="ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" containerName="istio-proxy" containerID="cri-o://2fc6c498419a8c7ac71003feb86893442646362a28a8000ba0b4a6210b18feb8" gracePeriod=30 Oct 29 23:35:44.484552 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:44.484524 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg"] Oct 29 23:35:44.485998 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:44.485422 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" podUID="6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" containerName="istio-proxy" containerID="cri-o://c1480c0611c17833c86ea4a30c1c2c4ab2e828ba32ad55bb2e66e86492bf09ee" gracePeriod=30 Oct 29 23:35:44.487149 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:44.487126 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2"] Oct 29 23:35:44.487372 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:44.487343 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" podUID="cb9f03d0-0cff-466b-9cc7-180db3286611" containerName="istio-proxy" containerID="cri-o://ce87eb8100b519fc9480b4f436fb93f941b018ecdc9e523746530e209846ca26" gracePeriod=30 Oct 29 23:35:44.556244 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:44.556206 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:45.825846 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:45.825816 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:48.004436 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:48.004401 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:35:48.004854 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:48.004453 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" podUID="ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:35:48.444779 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:48.444743 5056 patch_prober.go:29] interesting pod/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 29 23:35:48.444930 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:48.444805 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" podUID="6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 29 23:35:49.288856 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.288823 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-95ff59779-z9sbx"] Oct 29 23:35:49.289362 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.289048 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-95ff59779-z9sbx" podUID="4c8789da-b68f-46e9-b3e1-b380fd67a92b" containerName="manager" containerID="cri-o://96f6594ad4178e3447f89a7e8427bea6d7ef3a2db4d7af2c369284fa9ebd6d7e" gracePeriod=30 Oct 29 23:35:49.778292 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.769979 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-95ff59779-z9sbx" Oct 29 23:35:49.778292 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.773441 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:35:49.847521 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.847372 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:35:49.851569 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.851501 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:35:49.855587 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.855560 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:35:49.894855 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.894818 5056 generic.go:356] "Generic (PLEG): container finished" podID="ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" containerID="2fc6c498419a8c7ac71003feb86893442646362a28a8000ba0b4a6210b18feb8" exitCode=0 Oct 29 23:35:49.895018 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.894922 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" Oct 29 23:35:49.895018 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.894939 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" event={"ID":"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97","Type":"ContainerDied","Data":"2fc6c498419a8c7ac71003feb86893442646362a28a8000ba0b4a6210b18feb8"} Oct 29 23:35:49.895018 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.895006 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn" event={"ID":"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97","Type":"ContainerDied","Data":"60879c63633c3816588f30b525ba1e70f7c49749c9cc534b0a3fe3f73e6c9eb3"} Oct 29 23:35:49.895160 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.895041 5056 scope.go:122] "RemoveContainer" containerID="2fc6c498419a8c7ac71003feb86893442646362a28a8000ba0b4a6210b18feb8" Oct 29 23:35:49.896567 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.896543 5056 generic.go:356] "Generic (PLEG): container finished" podID="6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" containerID="c1480c0611c17833c86ea4a30c1c2c4ab2e828ba32ad55bb2e66e86492bf09ee" exitCode=0 Oct 29 23:35:49.896695 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.896583 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" Oct 29 23:35:49.896695 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.896629 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" event={"ID":"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6","Type":"ContainerDied","Data":"c1480c0611c17833c86ea4a30c1c2c4ab2e828ba32ad55bb2e66e86492bf09ee"} Oct 29 23:35:49.896695 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.896692 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg" event={"ID":"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6","Type":"ContainerDied","Data":"61da217ab1f461ce04e94106bb13b806018922eb447fcfca542ebfc7bfddf8d9"} Oct 29 23:35:49.898219 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.898199 5056 generic.go:356] "Generic (PLEG): container finished" podID="4c8789da-b68f-46e9-b3e1-b380fd67a92b" containerID="96f6594ad4178e3447f89a7e8427bea6d7ef3a2db4d7af2c369284fa9ebd6d7e" exitCode=0 Oct 29 23:35:49.898358 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.898297 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-95ff59779-z9sbx" Oct 29 23:35:49.898358 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.898302 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-95ff59779-z9sbx" event={"ID":"4c8789da-b68f-46e9-b3e1-b380fd67a92b","Type":"ContainerDied","Data":"96f6594ad4178e3447f89a7e8427bea6d7ef3a2db4d7af2c369284fa9ebd6d7e"} Oct 29 23:35:49.898358 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.898345 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-95ff59779-z9sbx" event={"ID":"4c8789da-b68f-46e9-b3e1-b380fd67a92b","Type":"ContainerDied","Data":"e4daa3b886302fcfd9b12791aeef83b5247d9c8314accfba988d6790aa40f6c1"} Oct 29 23:35:49.899755 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.899733 5056 generic.go:356] "Generic (PLEG): container finished" podID="cb9f03d0-0cff-466b-9cc7-180db3286611" containerID="ce87eb8100b519fc9480b4f436fb93f941b018ecdc9e523746530e209846ca26" exitCode=0 Oct 29 23:35:49.899857 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.899828 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" Oct 29 23:35:49.899958 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.899943 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" event={"ID":"cb9f03d0-0cff-466b-9cc7-180db3286611","Type":"ContainerDied","Data":"ce87eb8100b519fc9480b4f436fb93f941b018ecdc9e523746530e209846ca26"} Oct 29 23:35:49.900019 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.899979 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2" event={"ID":"cb9f03d0-0cff-466b-9cc7-180db3286611","Type":"ContainerDied","Data":"b65827220f754bb18d5abf9fc8f3564e8393ffe6ea742596c130fe066b51c516"} Oct 29 23:35:49.901691 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.901669 5056 generic.go:356] "Generic (PLEG): container finished" podID="80bda72d-4e80-497e-befe-514dcaff1e39" containerID="70ae58b3d3c9c312981eac19f538081c82c8fa9e43e6382a870fd8a69a040c05" exitCode=0 Oct 29 23:35:49.901857 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.901731 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" event={"ID":"80bda72d-4e80-497e-befe-514dcaff1e39","Type":"ContainerDied","Data":"70ae58b3d3c9c312981eac19f538081c82c8fa9e43e6382a870fd8a69a040c05"} Oct 29 23:35:49.901857 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.901765 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" event={"ID":"80bda72d-4e80-497e-befe-514dcaff1e39","Type":"ContainerDied","Data":"691983af408408090f2c023bc3298ca8b2eec012998682d4c0d60665e8c1a3d5"} Oct 29 23:35:49.901857 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.901814 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z" Oct 29 23:35:49.908403 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.908380 5056 scope.go:122] "RemoveContainer" containerID="2fc6c498419a8c7ac71003feb86893442646362a28a8000ba0b4a6210b18feb8" Oct 29 23:35:49.908687 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:35:49.908654 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fc6c498419a8c7ac71003feb86893442646362a28a8000ba0b4a6210b18feb8\": container with ID starting with 2fc6c498419a8c7ac71003feb86893442646362a28a8000ba0b4a6210b18feb8 not found: ID does not exist" containerID="2fc6c498419a8c7ac71003feb86893442646362a28a8000ba0b4a6210b18feb8" Oct 29 23:35:49.908754 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.908690 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fc6c498419a8c7ac71003feb86893442646362a28a8000ba0b4a6210b18feb8"} err="failed to get container status \"2fc6c498419a8c7ac71003feb86893442646362a28a8000ba0b4a6210b18feb8\": rpc error: code = NotFound desc = could not find container \"2fc6c498419a8c7ac71003feb86893442646362a28a8000ba0b4a6210b18feb8\": container with ID starting with 2fc6c498419a8c7ac71003feb86893442646362a28a8000ba0b4a6210b18feb8 not found: ID does not exist" Oct 29 23:35:49.908754 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.908713 5056 scope.go:122] "RemoveContainer" containerID="c1480c0611c17833c86ea4a30c1c2c4ab2e828ba32ad55bb2e66e86492bf09ee" Oct 29 23:35:49.917918 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.917897 5056 scope.go:122] "RemoveContainer" containerID="c1480c0611c17833c86ea4a30c1c2c4ab2e828ba32ad55bb2e66e86492bf09ee" Oct 29 23:35:49.918222 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:35:49.918181 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1480c0611c17833c86ea4a30c1c2c4ab2e828ba32ad55bb2e66e86492bf09ee\": container with ID starting with c1480c0611c17833c86ea4a30c1c2c4ab2e828ba32ad55bb2e66e86492bf09ee not found: ID does not exist" containerID="c1480c0611c17833c86ea4a30c1c2c4ab2e828ba32ad55bb2e66e86492bf09ee" Oct 29 23:35:49.918329 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.918234 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1480c0611c17833c86ea4a30c1c2c4ab2e828ba32ad55bb2e66e86492bf09ee"} err="failed to get container status \"c1480c0611c17833c86ea4a30c1c2c4ab2e828ba32ad55bb2e66e86492bf09ee\": rpc error: code = NotFound desc = could not find container \"c1480c0611c17833c86ea4a30c1c2c4ab2e828ba32ad55bb2e66e86492bf09ee\": container with ID starting with c1480c0611c17833c86ea4a30c1c2c4ab2e828ba32ad55bb2e66e86492bf09ee not found: ID does not exist" Oct 29 23:35:49.918329 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.918271 5056 scope.go:122] "RemoveContainer" containerID="96f6594ad4178e3447f89a7e8427bea6d7ef3a2db4d7af2c369284fa9ebd6d7e" Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.928082 5056 scope.go:122] "RemoveContainer" containerID="96f6594ad4178e3447f89a7e8427bea6d7ef3a2db4d7af2c369284fa9ebd6d7e" Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.928661 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-podinfo\") pod \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.928706 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istiod-ca-cert\") pod \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.928745 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-envoy\") pod \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.928785 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-podinfo\") pod \"cb9f03d0-0cff-466b-9cc7-180db3286611\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.928829 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c8789da-b68f-46e9-b3e1-b380fd67a92b-kube-api-access-mkctc\" (UniqueName: \"kubernetes.io/projected/4c8789da-b68f-46e9-b3e1-b380fd67a92b-kube-api-access-mkctc\") pod \"4c8789da-b68f-46e9-b3e1-b380fd67a92b\" (UID: \"4c8789da-b68f-46e9-b3e1-b380fd67a92b\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.928884 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4c8789da-b68f-46e9-b3e1-b380fd67a92b-webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c8789da-b68f-46e9-b3e1-b380fd67a92b-webhook-cert\") pod \"4c8789da-b68f-46e9-b3e1-b380fd67a92b\" (UID: \"4c8789da-b68f-46e9-b3e1-b380fd67a92b\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.928912 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-workload-socket\") pod \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.928945 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-kube-api-access-tjjdc\" (UniqueName: \"kubernetes.io/projected/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-kube-api-access-tjjdc\") pod \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.928977 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4c8789da-b68f-46e9-b3e1-b380fd67a92b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c8789da-b68f-46e9-b3e1-b380fd67a92b-metrics-certs\") pod \"4c8789da-b68f-46e9-b3e1-b380fd67a92b\" (UID: \"4c8789da-b68f-46e9-b3e1-b380fd67a92b\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.929025 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-credential-socket\") pod \"cb9f03d0-0cff-466b-9cc7-180db3286611\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.929081 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-data\") pod \"cb9f03d0-0cff-466b-9cc7-180db3286611\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.929112 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-credential-socket\") pod \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.929149 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-data\") pod \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.929177 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cb9f03d0-0cff-466b-9cc7-180db3286611-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb9f03d0-0cff-466b-9cc7-180db3286611-istiod-ca-cert\") pod \"cb9f03d0-0cff-466b-9cc7-180db3286611\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.929237 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-workload-socket\") pod \"cb9f03d0-0cff-466b-9cc7-180db3286611\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.929658 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-workload-certs\") pod \"cb9f03d0-0cff-466b-9cc7-180db3286611\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.929730 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb9f03d0-0cff-466b-9cc7-180db3286611-kube-api-access-lp4n8\" (UniqueName: \"kubernetes.io/projected/cb9f03d0-0cff-466b-9cc7-180db3286611-kube-api-access-lp4n8\") pod \"cb9f03d0-0cff-466b-9cc7-180db3286611\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.929760 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-token\" (UniqueName: \"kubernetes.io/projected/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-token\") pod \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.929786 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-workload-certs\") pod \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\" (UID: \"6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.929834 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-envoy\") pod \"cb9f03d0-0cff-466b-9cc7-180db3286611\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " Oct 29 23:35:49.930005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.929873 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-token\" (UniqueName: \"kubernetes.io/projected/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-token\") pod \"cb9f03d0-0cff-466b-9cc7-180db3286611\" (UID: \"cb9f03d0-0cff-466b-9cc7-180db3286611\") " Oct 29 23:35:49.930808 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.930575 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-credential-socket" pod "cb9f03d0-0cff-466b-9cc7-180db3286611" (UID: "cb9f03d0-0cff-466b-9cc7-180db3286611"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:49.930808 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:35:49.930662 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96f6594ad4178e3447f89a7e8427bea6d7ef3a2db4d7af2c369284fa9ebd6d7e\": container with ID starting with 96f6594ad4178e3447f89a7e8427bea6d7ef3a2db4d7af2c369284fa9ebd6d7e not found: ID does not exist" containerID="96f6594ad4178e3447f89a7e8427bea6d7ef3a2db4d7af2c369284fa9ebd6d7e" Oct 29 23:35:49.930808 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.930680 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-data" pod "cb9f03d0-0cff-466b-9cc7-180db3286611" (UID: "cb9f03d0-0cff-466b-9cc7-180db3286611"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:49.930808 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.930693 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96f6594ad4178e3447f89a7e8427bea6d7ef3a2db4d7af2c369284fa9ebd6d7e"} err="failed to get container status \"96f6594ad4178e3447f89a7e8427bea6d7ef3a2db4d7af2c369284fa9ebd6d7e\": rpc error: code = NotFound desc = could not find container \"96f6594ad4178e3447f89a7e8427bea6d7ef3a2db4d7af2c369284fa9ebd6d7e\": container with ID starting with 96f6594ad4178e3447f89a7e8427bea6d7ef3a2db4d7af2c369284fa9ebd6d7e not found: ID does not exist" Oct 29 23:35:49.930808 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.930716 5056 scope.go:122] "RemoveContainer" containerID="ce87eb8100b519fc9480b4f436fb93f941b018ecdc9e523746530e209846ca26" Oct 29 23:35:49.931047 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.930823 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-credential-socket" pod "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" (UID: "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:49.931047 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.931007 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-data" pod "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" (UID: "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:49.931320 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.931302 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb9f03d0-0cff-466b-9cc7-180db3286611-istiod-ca-cert" pod "cb9f03d0-0cff-466b-9cc7-180db3286611" (UID: "cb9f03d0-0cff-466b-9cc7-180db3286611"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:49.931491 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.931473 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-workload-socket" pod "cb9f03d0-0cff-466b-9cc7-180db3286611" (UID: "cb9f03d0-0cff-466b-9cc7-180db3286611"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:49.931610 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.931596 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-workload-certs" pod "cb9f03d0-0cff-466b-9cc7-180db3286611" (UID: "cb9f03d0-0cff-466b-9cc7-180db3286611"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:49.934982 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.934806 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-podinfo" pod "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" (UID: "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:35:49.935185 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.935130 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-workload-socket" pod "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" (UID: "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:49.935503 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.935465 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istiod-ca-cert" pod "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" (UID: "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:49.935838 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.935783 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c8789da-b68f-46e9-b3e1-b380fd67a92b-webhook-cert" pod "4c8789da-b68f-46e9-b3e1-b380fd67a92b" (UID: "4c8789da-b68f-46e9-b3e1-b380fd67a92b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:35:49.936095 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.936075 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-token" pod "cb9f03d0-0cff-466b-9cc7-180db3286611" (UID: "cb9f03d0-0cff-466b-9cc7-180db3286611"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:49.936492 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.936324 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-kube-api-access-tjjdc" pod "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" (UID: "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6"). InnerVolumeSpecName "kube-api-access-tjjdc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:49.936492 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.936425 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-podinfo" pod "cb9f03d0-0cff-466b-9cc7-180db3286611" (UID: "cb9f03d0-0cff-466b-9cc7-180db3286611"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:35:49.937179 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.937156 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c8789da-b68f-46e9-b3e1-b380fd67a92b-metrics-certs" pod "4c8789da-b68f-46e9-b3e1-b380fd67a92b" (UID: "4c8789da-b68f-46e9-b3e1-b380fd67a92b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:35:49.937580 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.937558 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-token" pod "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" (UID: "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:49.937649 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.937631 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-workload-certs" pod "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" (UID: "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:49.938232 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.938204 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb9f03d0-0cff-466b-9cc7-180db3286611-kube-api-access-lp4n8" pod "cb9f03d0-0cff-466b-9cc7-180db3286611" (UID: "cb9f03d0-0cff-466b-9cc7-180db3286611"). InnerVolumeSpecName "kube-api-access-lp4n8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:49.938869 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.938839 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c8789da-b68f-46e9-b3e1-b380fd67a92b-kube-api-access-mkctc" pod "4c8789da-b68f-46e9-b3e1-b380fd67a92b" (UID: "4c8789da-b68f-46e9-b3e1-b380fd67a92b"). InnerVolumeSpecName "kube-api-access-mkctc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:49.940199 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.940166 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-envoy" pod "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" (UID: "6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:49.940346 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.940299 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-envoy" pod "cb9f03d0-0cff-466b-9cc7-180db3286611" (UID: "cb9f03d0-0cff-466b-9cc7-180db3286611"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:49.949069 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.949047 5056 scope.go:122] "RemoveContainer" containerID="ce87eb8100b519fc9480b4f436fb93f941b018ecdc9e523746530e209846ca26" Oct 29 23:35:49.949441 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:35:49.949410 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce87eb8100b519fc9480b4f436fb93f941b018ecdc9e523746530e209846ca26\": container with ID starting with ce87eb8100b519fc9480b4f436fb93f941b018ecdc9e523746530e209846ca26 not found: ID does not exist" containerID="ce87eb8100b519fc9480b4f436fb93f941b018ecdc9e523746530e209846ca26" Oct 29 23:35:49.949583 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.949448 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce87eb8100b519fc9480b4f436fb93f941b018ecdc9e523746530e209846ca26"} err="failed to get container status \"ce87eb8100b519fc9480b4f436fb93f941b018ecdc9e523746530e209846ca26\": rpc error: code = NotFound desc = could not find container \"ce87eb8100b519fc9480b4f436fb93f941b018ecdc9e523746530e209846ca26\": container with ID starting with ce87eb8100b519fc9480b4f436fb93f941b018ecdc9e523746530e209846ca26 not found: ID does not exist" Oct 29 23:35:49.949583 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.949463 5056 scope.go:122] "RemoveContainer" containerID="70ae58b3d3c9c312981eac19f538081c82c8fa9e43e6382a870fd8a69a040c05" Oct 29 23:35:49.958010 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.957979 5056 scope.go:122] "RemoveContainer" containerID="70ae58b3d3c9c312981eac19f538081c82c8fa9e43e6382a870fd8a69a040c05" Oct 29 23:35:49.958311 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:35:49.958287 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70ae58b3d3c9c312981eac19f538081c82c8fa9e43e6382a870fd8a69a040c05\": container with ID starting with 70ae58b3d3c9c312981eac19f538081c82c8fa9e43e6382a870fd8a69a040c05 not found: ID does not exist" containerID="70ae58b3d3c9c312981eac19f538081c82c8fa9e43e6382a870fd8a69a040c05" Oct 29 23:35:49.958388 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:49.958321 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70ae58b3d3c9c312981eac19f538081c82c8fa9e43e6382a870fd8a69a040c05"} err="failed to get container status \"70ae58b3d3c9c312981eac19f538081c82c8fa9e43e6382a870fd8a69a040c05\": rpc error: code = NotFound desc = could not find container \"70ae58b3d3c9c312981eac19f538081c82c8fa9e43e6382a870fd8a69a040c05\": container with ID starting with 70ae58b3d3c9c312981eac19f538081c82c8fa9e43e6382a870fd8a69a040c05 not found: ID does not exist" Oct 29 23:35:50.031334 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031297 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/80bda72d-4e80-497e-befe-514dcaff1e39-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80bda72d-4e80-497e-befe-514dcaff1e39-istio-podinfo\") pod \"80bda72d-4e80-497e-befe-514dcaff1e39\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " Oct 29 23:35:50.031334 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031335 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80bda72d-4e80-497e-befe-514dcaff1e39-istio-token\" (UniqueName: \"kubernetes.io/projected/80bda72d-4e80-497e-befe-514dcaff1e39-istio-token\") pod \"80bda72d-4e80-497e-befe-514dcaff1e39\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " Oct 29 23:35:50.031565 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031356 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-istio-data\") pod \"80bda72d-4e80-497e-befe-514dcaff1e39\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " Oct 29 23:35:50.031565 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031401 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-workload-certs\") pod \"80bda72d-4e80-497e-befe-514dcaff1e39\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " Oct 29 23:35:50.031565 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031420 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-kube-api-access-fqm72\" (UniqueName: \"kubernetes.io/projected/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-kube-api-access-fqm72\") pod \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " Oct 29 23:35:50.031565 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031443 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-istio-envoy\") pod \"80bda72d-4e80-497e-befe-514dcaff1e39\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " Oct 29 23:35:50.031565 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031468 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-workload-socket\") pod \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " Oct 29 23:35:50.031565 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031487 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istiod-ca-cert\") pod \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " Oct 29 23:35:50.031565 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031518 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-workload-certs\") pod \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " Oct 29 23:35:50.031565 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031550 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-data\") pod \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " Oct 29 23:35:50.031993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031577 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/80bda72d-4e80-497e-befe-514dcaff1e39-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80bda72d-4e80-497e-befe-514dcaff1e39-istiod-ca-cert\") pod \"80bda72d-4e80-497e-befe-514dcaff1e39\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " Oct 29 23:35:50.031993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031648 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80bda72d-4e80-497e-befe-514dcaff1e39-kube-api-access-msg8f\" (UniqueName: \"kubernetes.io/projected/80bda72d-4e80-497e-befe-514dcaff1e39-kube-api-access-msg8f\") pod \"80bda72d-4e80-497e-befe-514dcaff1e39\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " Oct 29 23:35:50.031993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031682 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-podinfo\") pod \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " Oct 29 23:35:50.031993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031720 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-token\" (UniqueName: \"kubernetes.io/projected/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-token\") pod \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " Oct 29 23:35:50.031993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031758 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-credential-socket\") pod \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " Oct 29 23:35:50.031993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031785 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-credential-socket\") pod \"80bda72d-4e80-497e-befe-514dcaff1e39\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " Oct 29 23:35:50.031993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031842 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-workload-socket\") pod \"80bda72d-4e80-497e-befe-514dcaff1e39\" (UID: \"80bda72d-4e80-497e-befe-514dcaff1e39\") " Oct 29 23:35:50.031993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031873 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-envoy\") pod \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\" (UID: \"ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97\") " Oct 29 23:35:50.031993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.031876 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-workload-socket" pod "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" (UID: "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:50.032492 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032080 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-istio-data" pod "80bda72d-4e80-497e-befe-514dcaff1e39" (UID: "80bda72d-4e80-497e-befe-514dcaff1e39"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:50.032568 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032544 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkctc\" (UniqueName: \"kubernetes.io/projected/4c8789da-b68f-46e9-b3e1-b380fd67a92b-kube-api-access-mkctc\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032632 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032577 5056 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4c8789da-b68f-46e9-b3e1-b380fd67a92b-webhook-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032632 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032595 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032632 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032610 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjjdc\" (UniqueName: \"kubernetes.io/projected/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-kube-api-access-tjjdc\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032632 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032623 5056 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c8789da-b68f-46e9-b3e1-b380fd67a92b-metrics-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032637 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032650 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032663 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032677 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032691 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032705 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb9f03d0-0cff-466b-9cc7-180db3286611-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032719 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032734 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032747 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032761 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lp4n8\" (UniqueName: \"kubernetes.io/projected/cb9f03d0-0cff-466b-9cc7-180db3286611-kube-api-access-lp4n8\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032778 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032792 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032806 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.032836 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032833 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.033575 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032847 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.033575 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.033236 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.033575 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.033280 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.033575 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.033294 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb9f03d0-0cff-466b-9cc7-180db3286611-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.033575 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.032539 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-data" pod "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" (UID: "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:50.033575 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.033337 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istiod-ca-cert" pod "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" (UID: "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:50.033575 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.033515 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-workload-certs" pod "80bda72d-4e80-497e-befe-514dcaff1e39" (UID: "80bda72d-4e80-497e-befe-514dcaff1e39"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:50.033939 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.033616 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-workload-certs" pod "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" (UID: "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:50.033939 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.033831 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-credential-socket" pod "80bda72d-4e80-497e-befe-514dcaff1e39" (UID: "80bda72d-4e80-497e-befe-514dcaff1e39"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:50.034736 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.034575 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80bda72d-4e80-497e-befe-514dcaff1e39-istiod-ca-cert" pod "80bda72d-4e80-497e-befe-514dcaff1e39" (UID: "80bda72d-4e80-497e-befe-514dcaff1e39"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:35:50.034736 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.034672 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80bda72d-4e80-497e-befe-514dcaff1e39-istio-token" pod "80bda72d-4e80-497e-befe-514dcaff1e39" (UID: "80bda72d-4e80-497e-befe-514dcaff1e39"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:50.034929 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.034736 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-credential-socket" pod "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" (UID: "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:50.036213 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.036177 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-podinfo" pod "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" (UID: "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:35:50.037208 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.037182 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-kube-api-access-fqm72" pod "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" (UID: "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97"). InnerVolumeSpecName "kube-api-access-fqm72". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:50.037696 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.037665 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/80bda72d-4e80-497e-befe-514dcaff1e39-istio-podinfo" pod "80bda72d-4e80-497e-befe-514dcaff1e39" (UID: "80bda72d-4e80-497e-befe-514dcaff1e39"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 29 23:35:50.037696 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.037673 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-envoy" pod "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" (UID: "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:50.037857 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.037676 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-token" pod "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" (UID: "ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:50.037992 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.037968 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80bda72d-4e80-497e-befe-514dcaff1e39-kube-api-access-msg8f" pod "80bda72d-4e80-497e-befe-514dcaff1e39" (UID: "80bda72d-4e80-497e-befe-514dcaff1e39"). InnerVolumeSpecName "kube-api-access-msg8f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:35:50.038781 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.038757 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-workload-socket" pod "80bda72d-4e80-497e-befe-514dcaff1e39" (UID: "80bda72d-4e80-497e-befe-514dcaff1e39"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:50.039052 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.039026 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-istio-envoy" pod "80bda72d-4e80-497e-befe-514dcaff1e39" (UID: "80bda72d-4e80-497e-befe-514dcaff1e39"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:35:50.134907 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.134832 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.134907 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.134857 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.134907 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.134865 5056 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-data\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.134907 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.134874 5056 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/80bda72d-4e80-497e-befe-514dcaff1e39-istiod-ca-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.134907 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.134882 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msg8f\" (UniqueName: \"kubernetes.io/projected/80bda72d-4e80-497e-befe-514dcaff1e39-kube-api-access-msg8f\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.134907 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.134890 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.134907 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.134898 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.134907 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.134906 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.135243 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.134914 5056 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-credential-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.135243 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.134923 5056 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-workload-socket\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.135243 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.134931 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.135243 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.134939 5056 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/80bda72d-4e80-497e-befe-514dcaff1e39-istio-podinfo\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.135243 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.134946 5056 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/80bda72d-4e80-497e-befe-514dcaff1e39-istio-token\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.135243 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.134953 5056 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-workload-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.135243 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.134961 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqm72\" (UniqueName: \"kubernetes.io/projected/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97-kube-api-access-fqm72\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.135243 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.134969 5056 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/80bda72d-4e80-497e-befe-514dcaff1e39-istio-envoy\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:35:50.263695 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.263662 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg"] Oct 29 23:35:50.275227 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.275192 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-p8mwg"] Oct 29 23:35:50.308160 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.308128 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-95ff59779-z9sbx"] Oct 29 23:35:50.321065 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.315324 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-95ff59779-z9sbx"] Oct 29 23:35:50.350754 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.350711 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn"] Oct 29 23:35:50.356105 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.356070 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-f7fpn"] Oct 29 23:35:50.398144 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.398104 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z"] Oct 29 23:35:50.404986 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.404954 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-l629z"] Oct 29 23:35:50.444288 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.444229 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2"] Oct 29 23:35:50.452093 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.452065 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-aa7e96-openshift-default-7dd86477ff-2xtp2"] Oct 29 23:35:50.774006 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.773912 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c8789da-b68f-46e9-b3e1-b380fd67a92b" path="/var/lib/kubelet/pods/4c8789da-b68f-46e9-b3e1-b380fd67a92b/volumes" Oct 29 23:35:50.774584 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.774562 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" path="/var/lib/kubelet/pods/6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6/volumes" Oct 29 23:35:50.775191 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.775171 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="80bda72d-4e80-497e-befe-514dcaff1e39" path="/var/lib/kubelet/pods/80bda72d-4e80-497e-befe-514dcaff1e39/volumes" Oct 29 23:35:50.776228 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.776205 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb9f03d0-0cff-466b-9cc7-180db3286611" path="/var/lib/kubelet/pods/cb9f03d0-0cff-466b-9cc7-180db3286611/volumes" Oct 29 23:35:50.776824 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.776803 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" path="/var/lib/kubelet/pods/ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97/volumes" Oct 29 23:35:50.997966 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.997925 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-95ff59779-mchrq"] Oct 29 23:35:50.998379 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998362 5056 cpu_manager.go:414] "containerMap: removing container" podUID="6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" containerName="istio-proxy" Oct 29 23:35:50.998379 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998378 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998390 5056 cpu_manager.go:414] "containerMap: removing container" podUID="8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998395 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998407 5056 cpu_manager.go:414] "containerMap: removing container" podUID="d8622875-ba05-4797-92ce-4e208a3a0540" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998412 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d8622875-ba05-4797-92ce-4e208a3a0540" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998422 5056 cpu_manager.go:414] "containerMap: removing container" podUID="7e1dad7e-beb0-49c7-be70-d99766042790" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998427 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e1dad7e-beb0-49c7-be70-d99766042790" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998434 5056 cpu_manager.go:414] "containerMap: removing container" podUID="6d812120-3b8e-46c4-88d7-febeea4f0670" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998439 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d812120-3b8e-46c4-88d7-febeea4f0670" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998446 5056 cpu_manager.go:414] "containerMap: removing container" podUID="b46bbd7f-4b97-4b30-b500-1c31f0507898" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998450 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b46bbd7f-4b97-4b30-b500-1c31f0507898" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998458 5056 cpu_manager.go:414] "containerMap: removing container" podUID="cb9f03d0-0cff-466b-9cc7-180db3286611" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998462 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb9f03d0-0cff-466b-9cc7-180db3286611" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998471 5056 cpu_manager.go:414] "containerMap: removing container" podUID="9369cf81-a466-4661-b370-e676e87abd4b" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998475 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9369cf81-a466-4661-b370-e676e87abd4b" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998481 5056 cpu_manager.go:414] "containerMap: removing container" podUID="da79ef4d-119b-4eed-870b-5e0fb48e60c8" containerName="main" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998485 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="da79ef4d-119b-4eed-870b-5e0fb48e60c8" containerName="main" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998492 5056 cpu_manager.go:414] "containerMap: removing container" podUID="013d55e5-af92-4585-b7c4-33f76c3dabd4" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998496 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="013d55e5-af92-4585-b7c4-33f76c3dabd4" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998505 5056 cpu_manager.go:414] "containerMap: removing container" podUID="1a24e9f0-98ce-4367-99b2-25653f0171ee" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998509 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a24e9f0-98ce-4367-99b2-25653f0171ee" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998514 5056 cpu_manager.go:414] "containerMap: removing container" podUID="76b760f1-81f7-460a-bb76-1319cd789c50" containerName="maas-api" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998518 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76b760f1-81f7-460a-bb76-1319cd789c50" containerName="maas-api" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998525 5056 cpu_manager.go:414] "containerMap: removing container" podUID="1a24e9f0-98ce-4367-99b2-25653f0171ee" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998529 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a24e9f0-98ce-4367-99b2-25653f0171ee" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998535 5056 cpu_manager.go:414] "containerMap: removing container" podUID="9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998539 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998551 5056 cpu_manager.go:414] "containerMap: removing container" podUID="b6e9182f-276e-4ca6-bf97-16a1cf21ba00" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998556 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b6e9182f-276e-4ca6-bf97-16a1cf21ba00" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998565 5056 cpu_manager.go:414] "containerMap: removing container" podUID="80bda72d-4e80-497e-befe-514dcaff1e39" containerName="istio-proxy" Oct 29 23:35:50.998549 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998570 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80bda72d-4e80-497e-befe-514dcaff1e39" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998578 5056 cpu_manager.go:414] "containerMap: removing container" podUID="4c8789da-b68f-46e9-b3e1-b380fd67a92b" containerName="manager" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998582 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c8789da-b68f-46e9-b3e1-b380fd67a92b" containerName="manager" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998589 5056 cpu_manager.go:414] "containerMap: removing container" podUID="e4050977-832e-4b94-9161-5a23e0e27721" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998593 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4050977-832e-4b94-9161-5a23e0e27721" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998599 5056 cpu_manager.go:414] "containerMap: removing container" podUID="e80bef80-0024-4113-b7f5-f43eded6d18e" containerName="maas-api" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998603 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e80bef80-0024-4113-b7f5-f43eded6d18e" containerName="maas-api" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998610 5056 cpu_manager.go:414] "containerMap: removing container" podUID="29e6194d-41e0-4e53-8698-32ace257af0e" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998614 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29e6194d-41e0-4e53-8698-32ace257af0e" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998622 5056 cpu_manager.go:414] "containerMap: removing container" podUID="ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998626 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998632 5056 cpu_manager.go:414] "containerMap: removing container" podUID="29e6194d-41e0-4e53-8698-32ace257af0e" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998636 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29e6194d-41e0-4e53-8698-32ace257af0e" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998642 5056 cpu_manager.go:414] "containerMap: removing container" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998646 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998653 5056 cpu_manager.go:414] "containerMap: removing container" podUID="c103618d-ee4d-4568-bdaf-6e1b486f9277" containerName="maas-api" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998656 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c103618d-ee4d-4568-bdaf-6e1b486f9277" containerName="maas-api" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998664 5056 cpu_manager.go:414] "containerMap: removing container" podUID="4cb6d795-205b-465f-91b0-fa184fa8fe44" containerName="payload-pre-processing" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998670 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4cb6d795-205b-465f-91b0-fa184fa8fe44" containerName="payload-pre-processing" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998734 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="6fb5ef72-5ab0-4f03-9e6e-a0cc964377f6" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998742 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="8474da7b-b2ae-442f-9d8e-c95b8b2f1c0a" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998749 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="d8622875-ba05-4797-92ce-4e208a3a0540" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998757 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e1dad7e-beb0-49c7-be70-d99766042790" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998764 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d812120-3b8e-46c4-88d7-febeea4f0670" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998770 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="b46bbd7f-4b97-4b30-b500-1c31f0507898" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998775 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb9f03d0-0cff-466b-9cc7-180db3286611" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998780 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="9369cf81-a466-4661-b370-e676e87abd4b" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998786 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="da79ef4d-119b-4eed-870b-5e0fb48e60c8" containerName="main" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998791 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="013d55e5-af92-4585-b7c4-33f76c3dabd4" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998796 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a24e9f0-98ce-4367-99b2-25653f0171ee" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998801 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="76b760f1-81f7-460a-bb76-1319cd789c50" containerName="maas-api" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998807 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a24e9f0-98ce-4367-99b2-25653f0171ee" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998812 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="9f88e972-ce97-4ecc-83dc-0b4a2c4bf850" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998818 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="b6e9182f-276e-4ca6-bf97-16a1cf21ba00" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998823 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="80bda72d-4e80-497e-befe-514dcaff1e39" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998829 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c8789da-b68f-46e9-b3e1-b380fd67a92b" containerName="manager" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998834 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4050977-832e-4b94-9161-5a23e0e27721" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998838 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="e80bef80-0024-4113-b7f5-f43eded6d18e" containerName="maas-api" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998844 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="29e6194d-41e0-4e53-8698-32ace257af0e" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998850 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce6ba1a5-4a39-4e33-957f-4e8b4c1c0e97" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998854 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="29e6194d-41e0-4e53-8698-32ace257af0e" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998859 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="a267a091-f7f9-4f00-bf02-5fd668eb27c4" containerName="main" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998865 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="c103618d-ee4d-4568-bdaf-6e1b486f9277" containerName="maas-api" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998870 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="b46bbd7f-4b97-4b30-b500-1c31f0507898" containerName="istio-proxy" Oct 29 23:35:51.000115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:50.998876 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="4cb6d795-205b-465f-91b0-fa184fa8fe44" containerName="payload-pre-processing" Oct 29 23:35:51.004137 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.004114 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-95ff59779-mchrq" Oct 29 23:35:51.026668 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.026594 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-95ff59779-mchrq"] Oct 29 23:35:51.039809 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.039621 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Oct 29 23:35:51.039809 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.039745 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Oct 29 23:35:51.039809 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.039773 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5mgl4\"" Oct 29 23:35:51.040100 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.039947 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Oct 29 23:35:51.149368 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.149326 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/372fea6f-db19-484b-9f9e-41948085b6d2-metrics-certs\") pod \"maas-controller-95ff59779-mchrq\" (UID: \"372fea6f-db19-484b-9f9e-41948085b6d2\") " pod="opendatahub/maas-controller-95ff59779-mchrq" Oct 29 23:35:51.149527 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.149399 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shjn4\" (UniqueName: \"kubernetes.io/projected/372fea6f-db19-484b-9f9e-41948085b6d2-kube-api-access-shjn4\") pod \"maas-controller-95ff59779-mchrq\" (UID: \"372fea6f-db19-484b-9f9e-41948085b6d2\") " pod="opendatahub/maas-controller-95ff59779-mchrq" Oct 29 23:35:51.149625 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.149599 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/372fea6f-db19-484b-9f9e-41948085b6d2-webhook-cert\") pod \"maas-controller-95ff59779-mchrq\" (UID: \"372fea6f-db19-484b-9f9e-41948085b6d2\") " pod="opendatahub/maas-controller-95ff59779-mchrq" Oct 29 23:35:51.252946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.252902 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/372fea6f-db19-484b-9f9e-41948085b6d2-metrics-certs\") pod \"maas-controller-95ff59779-mchrq\" (UID: \"372fea6f-db19-484b-9f9e-41948085b6d2\") " pod="opendatahub/maas-controller-95ff59779-mchrq" Oct 29 23:35:51.256784 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.256751 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shjn4\" (UniqueName: \"kubernetes.io/projected/372fea6f-db19-484b-9f9e-41948085b6d2-kube-api-access-shjn4\") pod \"maas-controller-95ff59779-mchrq\" (UID: \"372fea6f-db19-484b-9f9e-41948085b6d2\") " pod="opendatahub/maas-controller-95ff59779-mchrq" Oct 29 23:35:51.257665 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.257643 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/372fea6f-db19-484b-9f9e-41948085b6d2-webhook-cert\") pod \"maas-controller-95ff59779-mchrq\" (UID: \"372fea6f-db19-484b-9f9e-41948085b6d2\") " pod="opendatahub/maas-controller-95ff59779-mchrq" Oct 29 23:35:51.259946 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.259910 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/372fea6f-db19-484b-9f9e-41948085b6d2-metrics-certs\") pod \"maas-controller-95ff59779-mchrq\" (UID: \"372fea6f-db19-484b-9f9e-41948085b6d2\") " pod="opendatahub/maas-controller-95ff59779-mchrq" Oct 29 23:35:51.262301 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.260761 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/372fea6f-db19-484b-9f9e-41948085b6d2-webhook-cert\") pod \"maas-controller-95ff59779-mchrq\" (UID: \"372fea6f-db19-484b-9f9e-41948085b6d2\") " pod="opendatahub/maas-controller-95ff59779-mchrq" Oct 29 23:35:51.288928 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.288867 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shjn4\" (UniqueName: \"kubernetes.io/projected/372fea6f-db19-484b-9f9e-41948085b6d2-kube-api-access-shjn4\") pod \"maas-controller-95ff59779-mchrq\" (UID: \"372fea6f-db19-484b-9f9e-41948085b6d2\") " pod="opendatahub/maas-controller-95ff59779-mchrq" Oct 29 23:35:51.321485 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.321451 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-95ff59779-mchrq" Oct 29 23:35:51.736291 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.733822 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-95ff59779-mchrq"] Oct 29 23:35:51.808029 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.808007 5056 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 29 23:35:51.981225 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:51.981124 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-95ff59779-mchrq" event={"ID":"372fea6f-db19-484b-9f9e-41948085b6d2","Type":"ContainerStarted","Data":"2952227d92af1d53f20b315f03bbdf383350eee2c1ff4be674a8c2f97a44e223"} Oct 29 23:35:52.510497 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:35:52.510451 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:35:52.994134 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:52.993431 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-95ff59779-mchrq" event={"ID":"372fea6f-db19-484b-9f9e-41948085b6d2","Type":"ContainerStarted","Data":"3900b42fb6c52fe60753629f559f4bbcc1fd101b98814dfb227cb9844b87b862"} Oct 29 23:35:52.994134 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:52.993762 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-95ff59779-mchrq" Oct 29 23:35:53.042862 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:53.042807 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-95ff59779-mchrq" podStartSLOduration=2.517221152 podStartE2EDuration="3.04279249s" totalImagesPullingTime="525.571338ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:35:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:35:53.040597724 +0000 UTC m=+1380.407345371" watchObservedRunningTime="2026-10-29 23:35:53.04279249 +0000 UTC m=+1380.409540120" Oct 29 23:35:53.869143 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:53.866954 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:54.026337 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:54.026205 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:54.427594 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:54.427563 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:35:57.462230 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:35:57.462199 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:36:00.446719 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:00.446687 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:36:04.001306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:04.001271 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-95ff59779-mchrq" Oct 29 23:36:08.908273 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:08.908236 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x"] Oct 29 23:36:08.908686 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:08.908672 5056 cpu_manager.go:414] "containerMap: removing container" podUID="b46bbd7f-4b97-4b30-b500-1c31f0507898" containerName="istio-proxy" Oct 29 23:36:08.908730 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:08.908687 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b46bbd7f-4b97-4b30-b500-1c31f0507898" containerName="istio-proxy" Oct 29 23:36:08.913979 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:08.913959 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:08.948738 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:08.948706 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x"] Oct 29 23:36:09.049319 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.049255 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fm2c\" (UniqueName: \"kubernetes.io/projected/a114874c-df13-4ea7-8518-c5d8725a1cb2-kube-api-access-6fm2c\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.049500 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.049337 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a114874c-df13-4ea7-8518-c5d8725a1cb2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.049500 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.049358 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a114874c-df13-4ea7-8518-c5d8725a1cb2-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.049500 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.049420 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a114874c-df13-4ea7-8518-c5d8725a1cb2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.049500 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.049439 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a114874c-df13-4ea7-8518-c5d8725a1cb2-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.049649 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.049526 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a114874c-df13-4ea7-8518-c5d8725a1cb2-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.049649 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.049571 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a114874c-df13-4ea7-8518-c5d8725a1cb2-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.049649 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.049594 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a114874c-df13-4ea7-8518-c5d8725a1cb2-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.049649 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.049614 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a114874c-df13-4ea7-8518-c5d8725a1cb2-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.150274 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.150237 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a114874c-df13-4ea7-8518-c5d8725a1cb2-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.150470 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.150300 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a114874c-df13-4ea7-8518-c5d8725a1cb2-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.150470 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.150345 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fm2c\" (UniqueName: \"kubernetes.io/projected/a114874c-df13-4ea7-8518-c5d8725a1cb2-kube-api-access-6fm2c\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.150470 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.150370 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a114874c-df13-4ea7-8518-c5d8725a1cb2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.150470 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.150399 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a114874c-df13-4ea7-8518-c5d8725a1cb2-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.150689 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.150534 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a114874c-df13-4ea7-8518-c5d8725a1cb2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.150689 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.150571 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a114874c-df13-4ea7-8518-c5d8725a1cb2-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.150689 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.150636 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a114874c-df13-4ea7-8518-c5d8725a1cb2-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.150840 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.150700 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a114874c-df13-4ea7-8518-c5d8725a1cb2-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.150840 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.150709 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a114874c-df13-4ea7-8518-c5d8725a1cb2-credential-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.150972 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.150946 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a114874c-df13-4ea7-8518-c5d8725a1cb2-workload-socket\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.151033 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.151006 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a114874c-df13-4ea7-8518-c5d8725a1cb2-istio-data\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.151077 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.151044 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a114874c-df13-4ea7-8518-c5d8725a1cb2-workload-certs\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.151137 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.151118 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a114874c-df13-4ea7-8518-c5d8725a1cb2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.152657 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.152634 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a114874c-df13-4ea7-8518-c5d8725a1cb2-istio-envoy\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.152910 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.152892 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a114874c-df13-4ea7-8518-c5d8725a1cb2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.174800 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.174743 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a114874c-df13-4ea7-8518-c5d8725a1cb2-istio-token\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.174965 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.174946 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fm2c\" (UniqueName: \"kubernetes.io/projected/a114874c-df13-4ea7-8518-c5d8725a1cb2-kube-api-access-6fm2c\") pod \"maas-default-gateway-openshift-default-746ff8f49f-mj46x\" (UID: \"a114874c-df13-4ea7-8518-c5d8725a1cb2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.228457 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.228435 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:09.541673 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.541649 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x"] Oct 29 23:36:09.543130 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:36:09.543104 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda114874c_df13_4ea7_8518_c5d8725a1cb2.slice/crio-7d67bada7ef2c490020401cf4f75de53c012257ce9ee9884d14de064fb4b131d WatchSource:0}: Error finding container 7d67bada7ef2c490020401cf4f75de53c012257ce9ee9884d14de064fb4b131d: Status 404 returned error can't find the container with id 7d67bada7ef2c490020401cf4f75de53c012257ce9ee9884d14de064fb4b131d Oct 29 23:36:09.545392 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.545361 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:36:09.545460 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.545426 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:36:09.545460 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:09.545455 5056 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"} Oct 29 23:36:10.082796 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:10.082760 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" event={"ID":"a114874c-df13-4ea7-8518-c5d8725a1cb2","Type":"ContainerStarted","Data":"eead92a486a9f7a2cc093d3348edc7ba51f89dde83f8cda056badf5b89104601"} Oct 29 23:36:10.083180 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:10.082823 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" event={"ID":"a114874c-df13-4ea7-8518-c5d8725a1cb2","Type":"ContainerStarted","Data":"7d67bada7ef2c490020401cf4f75de53c012257ce9ee9884d14de064fb4b131d"} Oct 29 23:36:10.145178 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:10.145126 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" podStartSLOduration=2.145106991 podStartE2EDuration="2.145106991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:36:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:36:10.143979132 +0000 UTC m=+1397.510726762" watchObservedRunningTime="2026-10-29 23:36:10.145106991 +0000 UTC m=+1397.511854623" Oct 29 23:36:10.238573 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:10.238524 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:10.240389 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:10.240356 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-mj46x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 23:36:10.240556 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:10.240414 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" podUID="a114874c-df13-4ea7-8518-c5d8725a1cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 23:36:11.238690 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:11.238650 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-mj46x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 23:36:11.239090 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:11.238708 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" podUID="a114874c-df13-4ea7-8518-c5d8725a1cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 23:36:12.234286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:12.230430 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-mj46x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 23:36:12.234286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:12.230499 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" podUID="a114874c-df13-4ea7-8518-c5d8725a1cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 23:36:13.236285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:13.229721 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-mj46x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 23:36:13.236285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:13.229782 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" podUID="a114874c-df13-4ea7-8518-c5d8725a1cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 23:36:14.231692 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:14.231651 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-mj46x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 23:36:14.232005 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:14.231975 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" podUID="a114874c-df13-4ea7-8518-c5d8725a1cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 23:36:15.237287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:15.231602 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-mj46x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 23:36:15.237287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:15.231676 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" podUID="a114874c-df13-4ea7-8518-c5d8725a1cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 23:36:16.229747 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:16.229712 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-mj46x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body= Oct 29 23:36:16.229929 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:16.229770 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" podUID="a114874c-df13-4ea7-8518-c5d8725a1cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" Oct 29 23:36:18.230721 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:18.230678 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-mj46x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:36:18.231356 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:18.231317 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" podUID="a114874c-df13-4ea7-8518-c5d8725a1cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:36:19.273433 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:19.273395 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-mj46x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:36:19.273969 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:19.273452 5056 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" podUID="a114874c-df13-4ea7-8518-c5d8725a1cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:36:20.061938 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:20.061906 5056 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:20.062255 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:20.062222 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:20.063186 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:20.063166 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" Oct 29 23:36:22.419786 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:36:22.419750 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:36:26.461725 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:26.461688 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:36:26.551685 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:26.551646 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-95ff59779-mchrq"] Oct 29 23:36:26.551894 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:26.551852 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-95ff59779-mchrq" podUID="372fea6f-db19-484b-9f9e-41948085b6d2" containerName="manager" containerID="cri-o://3900b42fb6c52fe60753629f559f4bbcc1fd101b98814dfb227cb9844b87b862" gracePeriod=30 Oct 29 23:36:26.806702 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:26.806676 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-95ff59779-mchrq" Oct 29 23:36:26.861102 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:26.861068 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/372fea6f-db19-484b-9f9e-41948085b6d2-webhook-cert\" (UniqueName: \"kubernetes.io/secret/372fea6f-db19-484b-9f9e-41948085b6d2-webhook-cert\") pod \"372fea6f-db19-484b-9f9e-41948085b6d2\" (UID: \"372fea6f-db19-484b-9f9e-41948085b6d2\") " Oct 29 23:36:26.861322 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:26.861124 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/372fea6f-db19-484b-9f9e-41948085b6d2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/372fea6f-db19-484b-9f9e-41948085b6d2-metrics-certs\") pod \"372fea6f-db19-484b-9f9e-41948085b6d2\" (UID: \"372fea6f-db19-484b-9f9e-41948085b6d2\") " Oct 29 23:36:26.861322 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:26.861173 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/372fea6f-db19-484b-9f9e-41948085b6d2-kube-api-access-shjn4\" (UniqueName: \"kubernetes.io/projected/372fea6f-db19-484b-9f9e-41948085b6d2-kube-api-access-shjn4\") pod \"372fea6f-db19-484b-9f9e-41948085b6d2\" (UID: \"372fea6f-db19-484b-9f9e-41948085b6d2\") " Oct 29 23:36:26.863931 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:26.863895 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/372fea6f-db19-484b-9f9e-41948085b6d2-metrics-certs" pod "372fea6f-db19-484b-9f9e-41948085b6d2" (UID: "372fea6f-db19-484b-9f9e-41948085b6d2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:36:26.864288 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:26.864235 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/372fea6f-db19-484b-9f9e-41948085b6d2-kube-api-access-shjn4" pod "372fea6f-db19-484b-9f9e-41948085b6d2" (UID: "372fea6f-db19-484b-9f9e-41948085b6d2"). InnerVolumeSpecName "kube-api-access-shjn4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:36:26.865661 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:26.865632 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/372fea6f-db19-484b-9f9e-41948085b6d2-webhook-cert" pod "372fea6f-db19-484b-9f9e-41948085b6d2" (UID: "372fea6f-db19-484b-9f9e-41948085b6d2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:36:26.962765 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:26.962702 5056 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/372fea6f-db19-484b-9f9e-41948085b6d2-webhook-cert\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:36:26.962765 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:26.962756 5056 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/372fea6f-db19-484b-9f9e-41948085b6d2-metrics-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:36:26.963034 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:26.962778 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shjn4\" (UniqueName: \"kubernetes.io/projected/372fea6f-db19-484b-9f9e-41948085b6d2-kube-api-access-shjn4\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:36:27.187840 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:27.187808 5056 generic.go:356] "Generic (PLEG): container finished" podID="372fea6f-db19-484b-9f9e-41948085b6d2" containerID="3900b42fb6c52fe60753629f559f4bbcc1fd101b98814dfb227cb9844b87b862" exitCode=0 Oct 29 23:36:27.188021 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:27.187850 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-95ff59779-mchrq" event={"ID":"372fea6f-db19-484b-9f9e-41948085b6d2","Type":"ContainerDied","Data":"3900b42fb6c52fe60753629f559f4bbcc1fd101b98814dfb227cb9844b87b862"} Oct 29 23:36:27.188021 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:27.187857 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-95ff59779-mchrq" Oct 29 23:36:27.188021 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:27.187892 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-95ff59779-mchrq" event={"ID":"372fea6f-db19-484b-9f9e-41948085b6d2","Type":"ContainerDied","Data":"2952227d92af1d53f20b315f03bbdf383350eee2c1ff4be674a8c2f97a44e223"} Oct 29 23:36:27.188021 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:27.187919 5056 scope.go:122] "RemoveContainer" containerID="3900b42fb6c52fe60753629f559f4bbcc1fd101b98814dfb227cb9844b87b862" Oct 29 23:36:27.197065 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:27.197046 5056 scope.go:122] "RemoveContainer" containerID="3900b42fb6c52fe60753629f559f4bbcc1fd101b98814dfb227cb9844b87b862" Oct 29 23:36:27.197348 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:36:27.197331 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3900b42fb6c52fe60753629f559f4bbcc1fd101b98814dfb227cb9844b87b862\": container with ID starting with 3900b42fb6c52fe60753629f559f4bbcc1fd101b98814dfb227cb9844b87b862 not found: ID does not exist" containerID="3900b42fb6c52fe60753629f559f4bbcc1fd101b98814dfb227cb9844b87b862" Oct 29 23:36:27.197414 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:27.197356 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3900b42fb6c52fe60753629f559f4bbcc1fd101b98814dfb227cb9844b87b862"} err="failed to get container status \"3900b42fb6c52fe60753629f559f4bbcc1fd101b98814dfb227cb9844b87b862\": rpc error: code = NotFound desc = could not find container \"3900b42fb6c52fe60753629f559f4bbcc1fd101b98814dfb227cb9844b87b862\": container with ID starting with 3900b42fb6c52fe60753629f559f4bbcc1fd101b98814dfb227cb9844b87b862 not found: ID does not exist" Oct 29 23:36:27.242235 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:27.242198 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-95ff59779-mchrq"] Oct 29 23:36:27.246183 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:27.246159 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-95ff59779-mchrq"] Oct 29 23:36:28.356832 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.356788 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:36:28.454491 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.454451 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-95ff59779-qx97f"] Oct 29 23:36:28.455127 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.455112 5056 cpu_manager.go:414] "containerMap: removing container" podUID="372fea6f-db19-484b-9f9e-41948085b6d2" containerName="manager" Oct 29 23:36:28.455175 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.455129 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="372fea6f-db19-484b-9f9e-41948085b6d2" containerName="manager" Oct 29 23:36:28.455253 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.455242 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="372fea6f-db19-484b-9f9e-41948085b6d2" containerName="manager" Oct 29 23:36:28.460348 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.460328 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-95ff59779-qx97f" Oct 29 23:36:28.477629 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.477605 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2872686b-a6b2-4f41-9c67-c3a095753570-metrics-certs\") pod \"maas-controller-95ff59779-qx97f\" (UID: \"2872686b-a6b2-4f41-9c67-c3a095753570\") " pod="opendatahub/maas-controller-95ff59779-qx97f" Oct 29 23:36:28.477748 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.477729 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx92c\" (UniqueName: \"kubernetes.io/projected/2872686b-a6b2-4f41-9c67-c3a095753570-kube-api-access-nx92c\") pod \"maas-controller-95ff59779-qx97f\" (UID: \"2872686b-a6b2-4f41-9c67-c3a095753570\") " pod="opendatahub/maas-controller-95ff59779-qx97f" Oct 29 23:36:28.477808 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.477794 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2872686b-a6b2-4f41-9c67-c3a095753570-webhook-cert\") pod \"maas-controller-95ff59779-qx97f\" (UID: \"2872686b-a6b2-4f41-9c67-c3a095753570\") " pod="opendatahub/maas-controller-95ff59779-qx97f" Oct 29 23:36:28.481220 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.481196 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Oct 29 23:36:28.497486 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.494348 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Oct 29 23:36:28.497486 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.494390 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Oct 29 23:36:28.497486 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.494394 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5mgl4\"" Oct 29 23:36:28.497688 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.497489 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-95ff59779-qx97f"] Oct 29 23:36:28.579061 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.579027 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2872686b-a6b2-4f41-9c67-c3a095753570-metrics-certs\") pod \"maas-controller-95ff59779-qx97f\" (UID: \"2872686b-a6b2-4f41-9c67-c3a095753570\") " pod="opendatahub/maas-controller-95ff59779-qx97f" Oct 29 23:36:28.579291 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.579112 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx92c\" (UniqueName: \"kubernetes.io/projected/2872686b-a6b2-4f41-9c67-c3a095753570-kube-api-access-nx92c\") pod \"maas-controller-95ff59779-qx97f\" (UID: \"2872686b-a6b2-4f41-9c67-c3a095753570\") " pod="opendatahub/maas-controller-95ff59779-qx97f" Oct 29 23:36:28.579291 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.579149 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2872686b-a6b2-4f41-9c67-c3a095753570-webhook-cert\") pod \"maas-controller-95ff59779-qx97f\" (UID: \"2872686b-a6b2-4f41-9c67-c3a095753570\") " pod="opendatahub/maas-controller-95ff59779-qx97f" Oct 29 23:36:28.581677 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.581646 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2872686b-a6b2-4f41-9c67-c3a095753570-webhook-cert\") pod \"maas-controller-95ff59779-qx97f\" (UID: \"2872686b-a6b2-4f41-9c67-c3a095753570\") " pod="opendatahub/maas-controller-95ff59779-qx97f" Oct 29 23:36:28.581778 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.581681 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2872686b-a6b2-4f41-9c67-c3a095753570-metrics-certs\") pod \"maas-controller-95ff59779-qx97f\" (UID: \"2872686b-a6b2-4f41-9c67-c3a095753570\") " pod="opendatahub/maas-controller-95ff59779-qx97f" Oct 29 23:36:28.604688 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.604665 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx92c\" (UniqueName: \"kubernetes.io/projected/2872686b-a6b2-4f41-9c67-c3a095753570-kube-api-access-nx92c\") pod \"maas-controller-95ff59779-qx97f\" (UID: \"2872686b-a6b2-4f41-9c67-c3a095753570\") " pod="opendatahub/maas-controller-95ff59779-qx97f" Oct 29 23:36:28.769393 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.769357 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="372fea6f-db19-484b-9f9e-41948085b6d2" path="/var/lib/kubelet/pods/372fea6f-db19-484b-9f9e-41948085b6d2/volumes" Oct 29 23:36:28.773569 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:28.773543 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-95ff59779-qx97f" Oct 29 23:36:29.074138 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:29.074116 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-95ff59779-qx97f"] Oct 29 23:36:29.075315 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:36:29.075289 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2872686b_a6b2_4f41_9c67_c3a095753570.slice/crio-af4d506cb31cfcf82b74a269c15f1907ec5ba6b2669c89e30f8281fd506d803b WatchSource:0}: Error finding container af4d506cb31cfcf82b74a269c15f1907ec5ba6b2669c89e30f8281fd506d803b: Status 404 returned error can't find the container with id af4d506cb31cfcf82b74a269c15f1907ec5ba6b2669c89e30f8281fd506d803b Oct 29 23:36:29.199749 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:29.199712 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-95ff59779-qx97f" event={"ID":"2872686b-a6b2-4f41-9c67-c3a095753570","Type":"ContainerStarted","Data":"af4d506cb31cfcf82b74a269c15f1907ec5ba6b2669c89e30f8281fd506d803b"} Oct 29 23:36:30.205879 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:30.205843 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-95ff59779-qx97f" event={"ID":"2872686b-a6b2-4f41-9c67-c3a095753570","Type":"ContainerStarted","Data":"aa7b36fa3c3df50b05b4c25db5da0b1fa28ab67a338b94f5e22a4409171aa854"} Oct 29 23:36:30.206298 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:30.205900 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-95ff59779-qx97f" Oct 29 23:36:30.252199 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:30.252145 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-95ff59779-qx97f" podStartSLOduration=1.916695262 podStartE2EDuration="2.252130528s" totalImagesPullingTime="335.435266ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:36:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:36:30.251552015 +0000 UTC m=+1417.618299647" watchObservedRunningTime="2026-10-29 23:36:30.252130528 +0000 UTC m=+1417.618878159" Oct 29 23:36:34.509426 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:34.509393 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:36:41.214153 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:41.214115 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-95ff59779-qx97f" Oct 29 23:36:45.918866 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:36:45.918831 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:36:52.416616 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:36:52.416577 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:37:01.984430 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:37:01.984388 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-nbfzf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:37:01.985115 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:37:01.985080 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" podUID="5adc898f-40d4-4851-af95-1f400d519360" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:37:16.721369 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:37:16.721326 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:37:16.887105 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:37:16.887068 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:37:22.583562 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:37:22.583509 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:37:46.057812 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:37:46.057730 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:37:52.422275 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:37:52.422228 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:37:53.035456 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:37:53.035416 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:37:53.037559 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:37:53.037533 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:38:21.103787 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:38:21.103414 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-mj46x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:38:21.103787 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:38:21.103480 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" podUID="a114874c-df13-4ea7-8518-c5d8725a1cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:38:21.676088 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:38:21.676023 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:38:22.415442 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:38:22.415406 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:38:36.113756 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:38:36.113715 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-mj46x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:38:36.114385 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:38:36.114353 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" podUID="a114874c-df13-4ea7-8518-c5d8725a1cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:38:46.104846 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:38:46.104812 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:38:46.263529 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:38:46.263495 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:38:52.415284 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:38:52.415226 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:38:53.588312 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:38:53.588254 5056 scope.go:122] "RemoveContainer" containerID="505ae531fa45e76620059bac8d471838e5109b77a039b9a8b6da01e7ee182194" Oct 29 23:39:22.579583 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:39:22.579523 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:39:46.282051 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:39:46.282019 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:39:48.661538 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:39:48.661503 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:39:48.832477 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:39:48.832438 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:39:52.415004 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:39:52.414967 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:40:07.532863 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:40:07.532798 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:40:07.748765 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:40:07.748725 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:40:22.414788 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:40:22.414748 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:40:46.395689 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:40:46.395608 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:40:52.415217 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:40:52.415174 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:41:00.979131 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:00.979096 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:01.461604 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:01.461566 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:01.619071 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:01.619027 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:05.051707 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:05.051670 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:05.391835 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:05.391804 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:06.341767 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.341736 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s"] Oct 29 23:41:06.345523 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.345502 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:41:06.370831 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.370650 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-d06645-serving-cert\"" Oct 29 23:41:06.375533 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.375504 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-k7tlz\"" Oct 29 23:41:06.380106 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.380076 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s"] Oct 29 23:41:06.406105 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.406074 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b34a52-8101-424c-9f77-79b37a10b89c-metrics-certs\") pod \"maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s\" (UID: \"e6b34a52-8101-424c-9f77-79b37a10b89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:41:06.406325 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.406125 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6b34a52-8101-424c-9f77-79b37a10b89c-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s\" (UID: \"e6b34a52-8101-424c-9f77-79b37a10b89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:41:06.406325 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.406221 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkdb5\" (UniqueName: \"kubernetes.io/projected/e6b34a52-8101-424c-9f77-79b37a10b89c-kube-api-access-hkdb5\") pod \"maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s\" (UID: \"e6b34a52-8101-424c-9f77-79b37a10b89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:41:06.406325 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.406282 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b34a52-8101-424c-9f77-79b37a10b89c-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s\" (UID: \"e6b34a52-8101-424c-9f77-79b37a10b89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:41:06.470772 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.470739 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:06.506859 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.506828 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b34a52-8101-424c-9f77-79b37a10b89c-metrics-certs\") pod \"maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s\" (UID: \"e6b34a52-8101-424c-9f77-79b37a10b89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:41:06.507035 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.506878 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6b34a52-8101-424c-9f77-79b37a10b89c-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s\" (UID: \"e6b34a52-8101-424c-9f77-79b37a10b89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:41:06.507035 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.506906 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkdb5\" (UniqueName: \"kubernetes.io/projected/e6b34a52-8101-424c-9f77-79b37a10b89c-kube-api-access-hkdb5\") pod \"maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s\" (UID: \"e6b34a52-8101-424c-9f77-79b37a10b89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:41:06.507035 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.506927 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b34a52-8101-424c-9f77-79b37a10b89c-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s\" (UID: \"e6b34a52-8101-424c-9f77-79b37a10b89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:41:06.507669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.507644 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b34a52-8101-424c-9f77-79b37a10b89c-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s\" (UID: \"e6b34a52-8101-424c-9f77-79b37a10b89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:41:06.509407 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.509378 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b34a52-8101-424c-9f77-79b37a10b89c-metrics-certs\") pod \"maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s\" (UID: \"e6b34a52-8101-424c-9f77-79b37a10b89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:41:06.509511 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.509408 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6b34a52-8101-424c-9f77-79b37a10b89c-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s\" (UID: \"e6b34a52-8101-424c-9f77-79b37a10b89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:41:06.531099 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.531074 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkdb5\" (UniqueName: \"kubernetes.io/projected/e6b34a52-8101-424c-9f77-79b37a10b89c-kube-api-access-hkdb5\") pod \"maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s\" (UID: \"e6b34a52-8101-424c-9f77-79b37a10b89c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:41:06.638114 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.638021 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:06.661247 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.661210 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:41:06.799097 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.799064 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:06.985103 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.985075 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s"] Oct 29 23:41:06.988566 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:41:06.988529 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6b34a52_8101_424c_9f77_79b37a10b89c.slice/crio-aa65697677b6c6e2e6dbe84c58aa3f2934ad3ddd632b68021a0efa240356c709 WatchSource:0}: Error finding container aa65697677b6c6e2e6dbe84c58aa3f2934ad3ddd632b68021a0efa240356c709: Status 404 returned error can't find the container with id aa65697677b6c6e2e6dbe84c58aa3f2934ad3ddd632b68021a0efa240356c709 Oct 29 23:41:06.990478 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:06.990455 5056 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 29 23:41:07.972517 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:07.972479 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" event={"ID":"e6b34a52-8101-424c-9f77-79b37a10b89c","Type":"ContainerStarted","Data":"76aa84f3220588c62b94a901b573a1330d5d7abe7e694a644f2c08fb3f456449"} Oct 29 23:41:07.972932 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:07.972533 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" event={"ID":"e6b34a52-8101-424c-9f77-79b37a10b89c","Type":"ContainerStarted","Data":"aa65697677b6c6e2e6dbe84c58aa3f2934ad3ddd632b68021a0efa240356c709"} Oct 29 23:41:07.972932 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:07.972557 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:41:08.017015 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:08.016958 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" podStartSLOduration=1.703917155 podStartE2EDuration="2.016939289s" totalImagesPullingTime="313.022134ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:41:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:41:08.016688646 +0000 UTC m=+1695.383436277" watchObservedRunningTime="2026-10-29 23:41:08.016939289 +0000 UTC m=+1695.383686921" Oct 29 23:41:08.328796 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:08.326544 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:09.867729 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:09.867683 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:11.579605 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:11.579491 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:13.340468 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:13.340429 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:13.983789 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:13.983757 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:41:14.917756 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:14.917717 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:16.670286 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:16.670222 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:18.310382 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:18.310345 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:19.781849 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:19.781816 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:19.939218 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:19.939184 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:20.418348 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:20.418314 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:20.602559 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:20.602526 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:21.540708 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:21.540669 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:22.416643 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:41:22.416575 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:41:22.565886 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:22.565843 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:23.327944 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:23.327900 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:25.061787 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:25.061730 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:26.689763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:26.689727 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:28.256877 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:28.256820 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:30.008571 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:30.008528 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:31.650244 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:31.650189 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:33.208511 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:33.208477 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:34.918294 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:34.918240 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:36.539923 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:36.539890 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:38.228947 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:38.228911 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:39.852385 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:39.852345 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:41.571733 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:41.571696 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:43.209554 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:43.209511 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:44.872191 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:44.872154 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:46.472841 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:46.472804 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:46.649046 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:46.649009 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:48.178397 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:48.178354 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:49.806738 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:49.806676 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:51.478462 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:51.478414 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:52.416110 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:41:52.416076 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:41:53.004295 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:53.004237 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:54.543409 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:54.543364 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:56.215787 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:56.215751 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:56.684458 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:56.684417 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:58.034015 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:58.033940 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:41:59.864295 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:41:59.863130 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:01.479662 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:01.479623 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:01.720813 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:01.720775 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:03.019433 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:03.019397 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:04.553638 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:04.553387 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:06.202874 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:06.202839 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:07.474063 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:07.474024 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:07.633090 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:07.633024 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:07.790564 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:07.790448 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:09.090219 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.090174 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s"] Oct 29 23:42:09.090695 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.090440 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" podUID="e6b34a52-8101-424c-9f77-79b37a10b89c" containerName="maas-api" containerID="cri-o://76aa84f3220588c62b94a901b573a1330d5d7abe7e694a644f2c08fb3f456449" gracePeriod=30 Oct 29 23:42:09.216933 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.216904 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:09.581370 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.581339 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:42:09.711555 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.711518 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:09.712623 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.712602 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e6b34a52-8101-424c-9f77-79b37a10b89c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b34a52-8101-424c-9f77-79b37a10b89c-metrics-certs\") pod \"e6b34a52-8101-424c-9f77-79b37a10b89c\" (UID: \"e6b34a52-8101-424c-9f77-79b37a10b89c\") " Oct 29 23:42:09.712713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.712643 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e6b34a52-8101-424c-9f77-79b37a10b89c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6b34a52-8101-424c-9f77-79b37a10b89c-maas-api-tls\") pod \"e6b34a52-8101-424c-9f77-79b37a10b89c\" (UID: \"e6b34a52-8101-424c-9f77-79b37a10b89c\") " Oct 29 23:42:09.712713 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.712699 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e6b34a52-8101-424c-9f77-79b37a10b89c-kube-api-access-hkdb5\" (UniqueName: \"kubernetes.io/projected/e6b34a52-8101-424c-9f77-79b37a10b89c-kube-api-access-hkdb5\") pod \"e6b34a52-8101-424c-9f77-79b37a10b89c\" (UID: \"e6b34a52-8101-424c-9f77-79b37a10b89c\") " Oct 29 23:42:09.712789 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.712749 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e6b34a52-8101-424c-9f77-79b37a10b89c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b34a52-8101-424c-9f77-79b37a10b89c-openshift-service-ca\") pod \"e6b34a52-8101-424c-9f77-79b37a10b89c\" (UID: \"e6b34a52-8101-424c-9f77-79b37a10b89c\") " Oct 29 23:42:09.713160 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.713133 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6b34a52-8101-424c-9f77-79b37a10b89c-openshift-service-ca" pod "e6b34a52-8101-424c-9f77-79b37a10b89c" (UID: "e6b34a52-8101-424c-9f77-79b37a10b89c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:42:09.714850 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.714812 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6b34a52-8101-424c-9f77-79b37a10b89c-metrics-certs" pod "e6b34a52-8101-424c-9f77-79b37a10b89c" (UID: "e6b34a52-8101-424c-9f77-79b37a10b89c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:42:09.714986 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.714905 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6b34a52-8101-424c-9f77-79b37a10b89c-maas-api-tls" pod "e6b34a52-8101-424c-9f77-79b37a10b89c" (UID: "e6b34a52-8101-424c-9f77-79b37a10b89c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:42:09.715191 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.715172 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6b34a52-8101-424c-9f77-79b37a10b89c-kube-api-access-hkdb5" pod "e6b34a52-8101-424c-9f77-79b37a10b89c" (UID: "e6b34a52-8101-424c-9f77-79b37a10b89c"). InnerVolumeSpecName "kube-api-access-hkdb5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:42:09.813895 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.813738 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkdb5\" (UniqueName: \"kubernetes.io/projected/e6b34a52-8101-424c-9f77-79b37a10b89c-kube-api-access-hkdb5\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:42:09.813895 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.813777 5056 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e6b34a52-8101-424c-9f77-79b37a10b89c-openshift-service-ca\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:42:09.813895 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.813794 5056 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e6b34a52-8101-424c-9f77-79b37a10b89c-metrics-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:42:09.813895 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.813808 5056 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e6b34a52-8101-424c-9f77-79b37a10b89c-maas-api-tls\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:42:09.871317 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:09.871288 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:10.314144 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:10.314106 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:10.318811 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:10.318782 5056 generic.go:356] "Generic (PLEG): container finished" podID="e6b34a52-8101-424c-9f77-79b37a10b89c" containerID="76aa84f3220588c62b94a901b573a1330d5d7abe7e694a644f2c08fb3f456449" exitCode=0 Oct 29 23:42:10.318966 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:10.318828 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" Oct 29 23:42:10.318966 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:10.318827 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" event={"ID":"e6b34a52-8101-424c-9f77-79b37a10b89c","Type":"ContainerDied","Data":"76aa84f3220588c62b94a901b573a1330d5d7abe7e694a644f2c08fb3f456449"} Oct 29 23:42:10.318966 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:10.318960 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s" event={"ID":"e6b34a52-8101-424c-9f77-79b37a10b89c","Type":"ContainerDied","Data":"aa65697677b6c6e2e6dbe84c58aa3f2934ad3ddd632b68021a0efa240356c709"} Oct 29 23:42:10.319134 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:10.318992 5056 scope.go:122] "RemoveContainer" containerID="76aa84f3220588c62b94a901b573a1330d5d7abe7e694a644f2c08fb3f456449" Oct 29 23:42:10.328731 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:10.328711 5056 scope.go:122] "RemoveContainer" containerID="76aa84f3220588c62b94a901b573a1330d5d7abe7e694a644f2c08fb3f456449" Oct 29 23:42:10.329019 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:42:10.328998 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76aa84f3220588c62b94a901b573a1330d5d7abe7e694a644f2c08fb3f456449\": container with ID starting with 76aa84f3220588c62b94a901b573a1330d5d7abe7e694a644f2c08fb3f456449 not found: ID does not exist" containerID="76aa84f3220588c62b94a901b573a1330d5d7abe7e694a644f2c08fb3f456449" Oct 29 23:42:10.329066 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:10.329028 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76aa84f3220588c62b94a901b573a1330d5d7abe7e694a644f2c08fb3f456449"} err="failed to get container status \"76aa84f3220588c62b94a901b573a1330d5d7abe7e694a644f2c08fb3f456449\": rpc error: code = NotFound desc = could not find container \"76aa84f3220588c62b94a901b573a1330d5d7abe7e694a644f2c08fb3f456449\": container with ID starting with 76aa84f3220588c62b94a901b573a1330d5d7abe7e694a644f2c08fb3f456449 not found: ID does not exist" Oct 29 23:42:10.373480 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:10.373446 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s"] Oct 29 23:42:10.386294 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:10.386252 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-d06645-8654b554fd-6tv2s"] Oct 29 23:42:10.769281 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:10.769239 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e6b34a52-8101-424c-9f77-79b37a10b89c" path="/var/lib/kubelet/pods/e6b34a52-8101-424c-9f77-79b37a10b89c/volumes" Oct 29 23:42:16.294431 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:16.294394 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:16.479427 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:16.479388 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:16.661086 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:16.661047 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:19.048529 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:19.048496 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:19.209851 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:19.209819 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:19.401546 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:19.401516 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:22.415630 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:42:22.415595 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:42:31.988426 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:31.988381 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-nbfzf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:42:31.989056 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:31.989018 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" podUID="5adc898f-40d4-4851-af95-1f400d519360" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:42:32.260933 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:32.260843 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:32.423787 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:32.423751 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:38.179521 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:38.179481 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:39.621218 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:39.621184 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:42.000678 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.000644 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq"] Oct 29 23:42:42.001163 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.000846 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" podUID="ba820266-433a-4903-bfa6-113a385e5327" containerName="manager" containerID="cri-o://cb91254fdd483400d84cf3e5c6cf5d8c4de0ec52eab7659543ebb60723d4b8c8" gracePeriod=10 Oct 29 23:42:42.260868 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.260800 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" Oct 29 23:42:42.395075 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.395047 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ba820266-433a-4903-bfa6-113a385e5327-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba820266-433a-4903-bfa6-113a385e5327-extensions-socket-volume\") pod \"ba820266-433a-4903-bfa6-113a385e5327\" (UID: \"ba820266-433a-4903-bfa6-113a385e5327\") " Oct 29 23:42:42.395254 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.395108 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ba820266-433a-4903-bfa6-113a385e5327-kube-api-access-crgr2\" (UniqueName: \"kubernetes.io/projected/ba820266-433a-4903-bfa6-113a385e5327-kube-api-access-crgr2\") pod \"ba820266-433a-4903-bfa6-113a385e5327\" (UID: \"ba820266-433a-4903-bfa6-113a385e5327\") " Oct 29 23:42:42.395455 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.395431 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba820266-433a-4903-bfa6-113a385e5327-extensions-socket-volume" pod "ba820266-433a-4903-bfa6-113a385e5327" (UID: "ba820266-433a-4903-bfa6-113a385e5327"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 29 23:42:42.397127 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.397107 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba820266-433a-4903-bfa6-113a385e5327-kube-api-access-crgr2" pod "ba820266-433a-4903-bfa6-113a385e5327" (UID: "ba820266-433a-4903-bfa6-113a385e5327"). InnerVolumeSpecName "kube-api-access-crgr2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:42:42.496122 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.496082 5056 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ba820266-433a-4903-bfa6-113a385e5327-extensions-socket-volume\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:42:42.496122 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.496116 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crgr2\" (UniqueName: \"kubernetes.io/projected/ba820266-433a-4903-bfa6-113a385e5327-kube-api-access-crgr2\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:42:42.544753 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.544659 5056 generic.go:356] "Generic (PLEG): container finished" podID="ba820266-433a-4903-bfa6-113a385e5327" containerID="cb91254fdd483400d84cf3e5c6cf5d8c4de0ec52eab7659543ebb60723d4b8c8" exitCode=0 Oct 29 23:42:42.544753 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.544705 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" Oct 29 23:42:42.544753 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.544742 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" event={"ID":"ba820266-433a-4903-bfa6-113a385e5327","Type":"ContainerDied","Data":"cb91254fdd483400d84cf3e5c6cf5d8c4de0ec52eab7659543ebb60723d4b8c8"} Oct 29 23:42:42.545010 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.544791 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq" event={"ID":"ba820266-433a-4903-bfa6-113a385e5327","Type":"ContainerDied","Data":"b5870afbb92dabcbbd5d5188ceb76f96c601602d49c4d7b91aac29b09c4e18ac"} Oct 29 23:42:42.545010 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.544819 5056 scope.go:122] "RemoveContainer" containerID="cb91254fdd483400d84cf3e5c6cf5d8c4de0ec52eab7659543ebb60723d4b8c8" Oct 29 23:42:42.555597 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.555571 5056 scope.go:122] "RemoveContainer" containerID="cb91254fdd483400d84cf3e5c6cf5d8c4de0ec52eab7659543ebb60723d4b8c8" Oct 29 23:42:42.555879 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:42:42.555861 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb91254fdd483400d84cf3e5c6cf5d8c4de0ec52eab7659543ebb60723d4b8c8\": container with ID starting with cb91254fdd483400d84cf3e5c6cf5d8c4de0ec52eab7659543ebb60723d4b8c8 not found: ID does not exist" containerID="cb91254fdd483400d84cf3e5c6cf5d8c4de0ec52eab7659543ebb60723d4b8c8" Oct 29 23:42:42.555962 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.555887 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb91254fdd483400d84cf3e5c6cf5d8c4de0ec52eab7659543ebb60723d4b8c8"} err="failed to get container status \"cb91254fdd483400d84cf3e5c6cf5d8c4de0ec52eab7659543ebb60723d4b8c8\": rpc error: code = NotFound desc = could not find container \"cb91254fdd483400d84cf3e5c6cf5d8c4de0ec52eab7659543ebb60723d4b8c8\": container with ID starting with cb91254fdd483400d84cf3e5c6cf5d8c4de0ec52eab7659543ebb60723d4b8c8 not found: ID does not exist" Oct 29 23:42:42.599746 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.599714 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq"] Oct 29 23:42:42.603951 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.603917 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-zkgpq"] Oct 29 23:42:42.770002 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:42.769972 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ba820266-433a-4903-bfa6-113a385e5327" path="/var/lib/kubelet/pods/ba820266-433a-4903-bfa6-113a385e5327/volumes" Oct 29 23:42:45.433034 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:45.432995 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:46.635077 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:46.635042 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:42:52.415405 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:42:52.415359 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:42:53.070418 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:53.070391 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:42:53.071229 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:42:53.071206 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:43:22.420786 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:43:22.420747 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:43:43.015588 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.015505 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-snhbq"] Oct 29 23:43:43.015993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.015956 5056 cpu_manager.go:414] "containerMap: removing container" podUID="ba820266-433a-4903-bfa6-113a385e5327" containerName="manager" Oct 29 23:43:43.015993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.015968 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ba820266-433a-4903-bfa6-113a385e5327" containerName="manager" Oct 29 23:43:43.015993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.015979 5056 cpu_manager.go:414] "containerMap: removing container" podUID="e6b34a52-8101-424c-9f77-79b37a10b89c" containerName="maas-api" Oct 29 23:43:43.015993 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.015984 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e6b34a52-8101-424c-9f77-79b37a10b89c" containerName="maas-api" Oct 29 23:43:43.016140 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.016071 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="ba820266-433a-4903-bfa6-113a385e5327" containerName="manager" Oct 29 23:43:43.016140 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.016080 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="e6b34a52-8101-424c-9f77-79b37a10b89c" containerName="maas-api" Oct 29 23:43:43.019382 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.019365 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-snhbq" Oct 29 23:43:43.047995 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.047969 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-9rhfr\"" Oct 29 23:43:43.079373 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.079342 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-snhbq"] Oct 29 23:43:43.090895 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.090865 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7xkk\" (UniqueName: \"kubernetes.io/projected/2ca7b491-54eb-444a-a35d-3ff844706209-kube-api-access-m7xkk\") pod \"kuadrant-operator-controller-manager-57bb9bfff-snhbq\" (UID: \"2ca7b491-54eb-444a-a35d-3ff844706209\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-snhbq" Oct 29 23:43:43.091041 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.091011 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ca7b491-54eb-444a-a35d-3ff844706209-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-snhbq\" (UID: \"2ca7b491-54eb-444a-a35d-3ff844706209\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-snhbq" Oct 29 23:43:43.191763 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.191730 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7xkk\" (UniqueName: \"kubernetes.io/projected/2ca7b491-54eb-444a-a35d-3ff844706209-kube-api-access-m7xkk\") pod \"kuadrant-operator-controller-manager-57bb9bfff-snhbq\" (UID: \"2ca7b491-54eb-444a-a35d-3ff844706209\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-snhbq" Oct 29 23:43:43.191948 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.191813 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ca7b491-54eb-444a-a35d-3ff844706209-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-snhbq\" (UID: \"2ca7b491-54eb-444a-a35d-3ff844706209\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-snhbq" Oct 29 23:43:43.192199 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.192180 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2ca7b491-54eb-444a-a35d-3ff844706209-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-57bb9bfff-snhbq\" (UID: \"2ca7b491-54eb-444a-a35d-3ff844706209\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-snhbq" Oct 29 23:43:43.216072 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.216042 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7xkk\" (UniqueName: \"kubernetes.io/projected/2ca7b491-54eb-444a-a35d-3ff844706209-kube-api-access-m7xkk\") pod \"kuadrant-operator-controller-manager-57bb9bfff-snhbq\" (UID: \"2ca7b491-54eb-444a-a35d-3ff844706209\") " pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-snhbq" Oct 29 23:43:43.332901 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.332815 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-snhbq" Oct 29 23:43:43.663023 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.662997 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-snhbq"] Oct 29 23:43:43.664990 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:43:43.664963 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ca7b491_54eb_444a_a35d_3ff844706209.slice/crio-ebc585b54e9c66c41b5423ea78b213eec66ee0adaa91d69aab50dbabc3b6d1e2 WatchSource:0}: Error finding container ebc585b54e9c66c41b5423ea78b213eec66ee0adaa91d69aab50dbabc3b6d1e2: Status 404 returned error can't find the container with id ebc585b54e9c66c41b5423ea78b213eec66ee0adaa91d69aab50dbabc3b6d1e2 Oct 29 23:43:43.843276 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.843210 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-snhbq" event={"ID":"2ca7b491-54eb-444a-a35d-3ff844706209","Type":"ContainerStarted","Data":"d3f727bfa166e8192a958ba4ade69ef2ab38d564d88a707a7ecf4340754cbad5"} Oct 29 23:43:43.843458 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.843304 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-snhbq" Oct 29 23:43:43.843458 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.843327 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-snhbq" event={"ID":"2ca7b491-54eb-444a-a35d-3ff844706209","Type":"ContainerStarted","Data":"ebc585b54e9c66c41b5423ea78b213eec66ee0adaa91d69aab50dbabc3b6d1e2"} Oct 29 23:43:43.893755 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:43.893702 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-snhbq" podStartSLOduration=1.8936886849999999 podStartE2EDuration="1.893688685s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:43:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:43:43.89300701 +0000 UTC m=+1851.259754652" watchObservedRunningTime="2026-10-29 23:43:43.893688685 +0000 UTC m=+1851.260436316" Oct 29 23:43:46.785814 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:46.785781 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:43:52.415439 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:43:52.415398 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:43:54.850817 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:43:54.850784 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-57bb9bfff-snhbq" Oct 29 23:44:03.942435 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:03.942390 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-blm5t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:44:03.943102 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:03.943051 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-blm5t" podUID="d81526c4-5f8d-4833-beac-6cc8507caba9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:44:04.363679 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:04.363580 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-9k9s2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:44:04.363970 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:04.363927 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" podUID="a27239f9-531e-4f44-be00-22c11b243a3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:44:22.415690 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:44:22.415654 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:44:32.527918 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:32.527876 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:33.079767 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:33.079731 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:33.234185 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:33.234118 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:36.249657 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:36.249626 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:36.429194 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:36.429151 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:37.642919 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.642885 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79"] Oct 29 23:44:37.646846 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.646829 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:44:37.681493 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.681464 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-06f91a-serving-cert\"" Oct 29 23:44:37.681675 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.681628 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-k7tlz\"" Oct 29 23:44:37.683515 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.683489 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79"] Oct 29 23:44:37.690076 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.690045 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fff4ec99-eefd-4c44-836e-0b42803e5055-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79\" (UID: \"fff4ec99-eefd-4c44-836e-0b42803e5055\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:44:37.690219 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.690091 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzfcc\" (UniqueName: \"kubernetes.io/projected/fff4ec99-eefd-4c44-836e-0b42803e5055-kube-api-access-vzfcc\") pod \"maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79\" (UID: \"fff4ec99-eefd-4c44-836e-0b42803e5055\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:44:37.690345 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.690236 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff4ec99-eefd-4c44-836e-0b42803e5055-metrics-certs\") pod \"maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79\" (UID: \"fff4ec99-eefd-4c44-836e-0b42803e5055\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:44:37.690418 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.690387 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff4ec99-eefd-4c44-836e-0b42803e5055-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79\" (UID: \"fff4ec99-eefd-4c44-836e-0b42803e5055\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:44:37.778329 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.778288 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:37.791638 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.791610 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fff4ec99-eefd-4c44-836e-0b42803e5055-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79\" (UID: \"fff4ec99-eefd-4c44-836e-0b42803e5055\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:44:37.791776 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.791645 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzfcc\" (UniqueName: \"kubernetes.io/projected/fff4ec99-eefd-4c44-836e-0b42803e5055-kube-api-access-vzfcc\") pod \"maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79\" (UID: \"fff4ec99-eefd-4c44-836e-0b42803e5055\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:44:37.791776 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.791735 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff4ec99-eefd-4c44-836e-0b42803e5055-metrics-certs\") pod \"maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79\" (UID: \"fff4ec99-eefd-4c44-836e-0b42803e5055\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:44:37.791776 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:44:37.791763 5056 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-serving-cert: secret "maas-api-e2e-worker-single-main-06f91a-serving-cert" not found Oct 29 23:44:37.791949 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:44:37.791821 5056 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fff4ec99-eefd-4c44-836e-0b42803e5055-maas-api-tls podName:fff4ec99-eefd-4c44-836e-0b42803e5055 nodeName:}" failed. No retries permitted until 2026-10-29 23:44:38.291802974 +0000 UTC m=+1905.658550603 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fff4ec99-eefd-4c44-836e-0b42803e5055-maas-api-tls") pod "maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" (UID: "fff4ec99-eefd-4c44-836e-0b42803e5055") : secret "maas-api-e2e-worker-single-main-06f91a-serving-cert" not found Oct 29 23:44:37.791949 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.791872 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff4ec99-eefd-4c44-836e-0b42803e5055-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79\" (UID: \"fff4ec99-eefd-4c44-836e-0b42803e5055\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:44:37.792602 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.792581 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff4ec99-eefd-4c44-836e-0b42803e5055-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79\" (UID: \"fff4ec99-eefd-4c44-836e-0b42803e5055\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:44:37.794046 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.794020 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff4ec99-eefd-4c44-836e-0b42803e5055-metrics-certs\") pod \"maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79\" (UID: \"fff4ec99-eefd-4c44-836e-0b42803e5055\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:44:37.822651 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.822587 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzfcc\" (UniqueName: \"kubernetes.io/projected/fff4ec99-eefd-4c44-836e-0b42803e5055-kube-api-access-vzfcc\") pod \"maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79\" (UID: \"fff4ec99-eefd-4c44-836e-0b42803e5055\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:44:37.988720 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:37.988644 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:38.164218 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:38.163874 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:38.295923 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:38.295849 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fff4ec99-eefd-4c44-836e-0b42803e5055-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79\" (UID: \"fff4ec99-eefd-4c44-836e-0b42803e5055\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:44:38.298298 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:38.298255 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fff4ec99-eefd-4c44-836e-0b42803e5055-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79\" (UID: \"fff4ec99-eefd-4c44-836e-0b42803e5055\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:44:38.559482 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:38.559404 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:44:38.861009 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:38.860983 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79"] Oct 29 23:44:38.863324 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:44:38.863288 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfff4ec99_eefd_4c44_836e_0b42803e5055.slice/crio-6e45bba36679af753a8f048b82b33ceafa97c1376aa7697a3cf4f3c5a3230cd8 WatchSource:0}: Error finding container 6e45bba36679af753a8f048b82b33ceafa97c1376aa7697a3cf4f3c5a3230cd8: Status 404 returned error can't find the container with id 6e45bba36679af753a8f048b82b33ceafa97c1376aa7697a3cf4f3c5a3230cd8 Oct 29 23:44:39.195030 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:39.194995 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" event={"ID":"fff4ec99-eefd-4c44-836e-0b42803e5055","Type":"ContainerStarted","Data":"6e45bba36679af753a8f048b82b33ceafa97c1376aa7697a3cf4f3c5a3230cd8"} Oct 29 23:44:39.557371 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:39.552753 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:40.201475 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:40.201427 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" event={"ID":"fff4ec99-eefd-4c44-836e-0b42803e5055","Type":"ContainerStarted","Data":"871f74bf8bc5c5a1a154be9c164a31ff3efa72ecf5094ef0c28324e6e49e2fce"} Oct 29 23:44:40.201966 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:40.201579 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:44:40.246011 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:40.245960 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" podStartSLOduration=2.886247657 podStartE2EDuration="3.245945316s" totalImagesPullingTime="359.697659ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:44:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:44:40.244928453 +0000 UTC m=+1907.611676085" watchObservedRunningTime="2026-10-29 23:44:40.245945316 +0000 UTC m=+1907.612692991" Oct 29 23:44:41.090872 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:41.090835 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:42.849237 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:42.849204 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:44.568100 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:44.568045 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:46.217478 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:46.217450 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:44:46.342577 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:46.342539 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:46.911729 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:46.911690 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:47.981581 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:47.981542 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:49.565108 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:49.565058 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:51.110618 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:51.110567 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:51.322706 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:51.322664 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:51.754385 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:51.754356 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:51.937002 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:51.936972 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:52.424803 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:44:52.424763 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:44:53.216134 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:53.216101 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:54.004554 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:54.004523 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:54.224563 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:54.224532 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:55.015638 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:55.015593 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:56.612116 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:56.612075 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:58.260630 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:58.260588 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:44:59.940287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:44:59.940226 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:01.606490 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:01.606453 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:03.162065 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:03.162027 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:04.716174 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:04.716070 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:06.338937 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:06.338899 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:08.049981 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:08.049935 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:09.674614 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:09.674581 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:11.393868 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:11.393834 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:13.074933 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:13.074893 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:14.704827 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:14.704786 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:16.241757 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:16.241717 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:17.873326 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:17.873253 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:19.630655 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:19.630613 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:21.172963 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:21.172926 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:22.416016 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:45:22.415977 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:45:22.695306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:22.695112 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:23.631356 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:23.631315 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:24.668822 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:24.668787 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:26.321619 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:26.321584 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:27.992680 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:27.992647 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:28.713583 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:28.713552 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:29.517386 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:29.517349 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:31.096897 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:31.096860 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:32.819213 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:32.819177 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:34.150359 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.150326 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79"] Oct 29 23:45:34.150811 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.150583 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" podUID="fff4ec99-eefd-4c44-836e-0b42803e5055" containerName="maas-api" containerID="cri-o://871f74bf8bc5c5a1a154be9c164a31ff3efa72ecf5094ef0c28324e6e49e2fce" gracePeriod=30 Oct 29 23:45:34.268479 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.268444 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:34.459685 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.459655 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:34.503951 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.503904 5056 generic.go:356] "Generic (PLEG): container finished" podID="fff4ec99-eefd-4c44-836e-0b42803e5055" containerID="871f74bf8bc5c5a1a154be9c164a31ff3efa72ecf5094ef0c28324e6e49e2fce" exitCode=0 Oct 29 23:45:34.504167 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.504045 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" event={"ID":"fff4ec99-eefd-4c44-836e-0b42803e5055","Type":"ContainerDied","Data":"871f74bf8bc5c5a1a154be9c164a31ff3efa72ecf5094ef0c28324e6e49e2fce"} Oct 29 23:45:34.606015 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.605988 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:45:34.650306 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.650197 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:34.703486 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.703404 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fff4ec99-eefd-4c44-836e-0b42803e5055-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fff4ec99-eefd-4c44-836e-0b42803e5055-maas-api-tls\") pod \"fff4ec99-eefd-4c44-836e-0b42803e5055\" (UID: \"fff4ec99-eefd-4c44-836e-0b42803e5055\") " Oct 29 23:45:34.703668 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.703514 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fff4ec99-eefd-4c44-836e-0b42803e5055-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff4ec99-eefd-4c44-836e-0b42803e5055-metrics-certs\") pod \"fff4ec99-eefd-4c44-836e-0b42803e5055\" (UID: \"fff4ec99-eefd-4c44-836e-0b42803e5055\") " Oct 29 23:45:34.703668 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.703574 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fff4ec99-eefd-4c44-836e-0b42803e5055-kube-api-access-vzfcc\" (UniqueName: \"kubernetes.io/projected/fff4ec99-eefd-4c44-836e-0b42803e5055-kube-api-access-vzfcc\") pod \"fff4ec99-eefd-4c44-836e-0b42803e5055\" (UID: \"fff4ec99-eefd-4c44-836e-0b42803e5055\") " Oct 29 23:45:34.703668 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.703622 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fff4ec99-eefd-4c44-836e-0b42803e5055-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff4ec99-eefd-4c44-836e-0b42803e5055-openshift-service-ca\") pod \"fff4ec99-eefd-4c44-836e-0b42803e5055\" (UID: \"fff4ec99-eefd-4c44-836e-0b42803e5055\") " Oct 29 23:45:34.704348 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.704316 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fff4ec99-eefd-4c44-836e-0b42803e5055-openshift-service-ca" pod "fff4ec99-eefd-4c44-836e-0b42803e5055" (UID: "fff4ec99-eefd-4c44-836e-0b42803e5055"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:45:34.705858 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.705822 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fff4ec99-eefd-4c44-836e-0b42803e5055-maas-api-tls" pod "fff4ec99-eefd-4c44-836e-0b42803e5055" (UID: "fff4ec99-eefd-4c44-836e-0b42803e5055"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:45:34.706050 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.706026 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fff4ec99-eefd-4c44-836e-0b42803e5055-metrics-certs" pod "fff4ec99-eefd-4c44-836e-0b42803e5055" (UID: "fff4ec99-eefd-4c44-836e-0b42803e5055"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:45:34.706214 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.706192 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fff4ec99-eefd-4c44-836e-0b42803e5055-kube-api-access-vzfcc" pod "fff4ec99-eefd-4c44-836e-0b42803e5055" (UID: "fff4ec99-eefd-4c44-836e-0b42803e5055"). InnerVolumeSpecName "kube-api-access-vzfcc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:45:34.804918 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.804891 5056 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fff4ec99-eefd-4c44-836e-0b42803e5055-metrics-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:45:34.804918 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.804917 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzfcc\" (UniqueName: \"kubernetes.io/projected/fff4ec99-eefd-4c44-836e-0b42803e5055-kube-api-access-vzfcc\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:45:34.805082 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.804927 5056 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fff4ec99-eefd-4c44-836e-0b42803e5055-openshift-service-ca\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:45:34.805082 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:34.804935 5056 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fff4ec99-eefd-4c44-836e-0b42803e5055-maas-api-tls\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:45:35.510957 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:35.510921 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" Oct 29 23:45:35.510957 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:35.510942 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79" event={"ID":"fff4ec99-eefd-4c44-836e-0b42803e5055","Type":"ContainerDied","Data":"6e45bba36679af753a8f048b82b33ceafa97c1376aa7697a3cf4f3c5a3230cd8"} Oct 29 23:45:35.511655 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:35.511004 5056 scope.go:122] "RemoveContainer" containerID="871f74bf8bc5c5a1a154be9c164a31ff3efa72ecf5094ef0c28324e6e49e2fce" Oct 29 23:45:35.568249 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:35.568205 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79"] Oct 29 23:45:35.581218 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:35.581186 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-06f91a-856856477b-9rt79"] Oct 29 23:45:35.632949 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:35.632917 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:35.794122 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:35.794032 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:36.299892 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:36.299811 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:36.769905 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:36.769872 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fff4ec99-eefd-4c44-836e-0b42803e5055" path="/var/lib/kubelet/pods/fff4ec99-eefd-4c44-836e-0b42803e5055/volumes" Oct 29 23:45:42.556468 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:42.556435 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:42.744710 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:42.744671 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:42.906781 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:42.906631 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:46.988940 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:46.988897 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:47.489627 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:45:47.489566 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:45:52.573667 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:45:52.573625 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:46:03.572464 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:46:03.572428 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:46:06.409852 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:46:06.409818 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:46:06.578742 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:46:06.578699 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:46:12.879569 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:46:12.879499 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:46:21.654439 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:46:21.654399 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:46:22.415273 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:46:22.415220 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:46:37.215381 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:46:37.215279 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:46:43.207680 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:46:43.207640 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:46:46.989297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:46:46.987345 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-nbfzf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:46:46.989297 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:46:46.987422 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-nbfzf" podUID="5adc898f-40d4-4851-af95-1f400d519360" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:46:47.190068 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:46:47.190009 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:46:47.350562 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:46:47.350478 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:46:52.423309 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:46:52.423244 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:46:54.135075 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:46:54.135043 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:47:06.108285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:06.108075 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-mj46x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:47:06.108285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:06.108158 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-mj46x" podUID="a114874c-df13-4ea7-8518-c5d8725a1cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:47:19.410004 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.409970 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-fj5sh"] Oct 29 23:47:19.410519 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.410433 5056 cpu_manager.go:414] "containerMap: removing container" podUID="fff4ec99-eefd-4c44-836e-0b42803e5055" containerName="maas-api" Oct 29 23:47:19.410519 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.410444 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fff4ec99-eefd-4c44-836e-0b42803e5055" containerName="maas-api" Oct 29 23:47:19.410591 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.410535 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="fff4ec99-eefd-4c44-836e-0b42803e5055" containerName="maas-api" Oct 29 23:47:19.413724 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.413702 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" Oct 29 23:47:19.446682 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.446649 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-fj5sh"] Oct 29 23:47:19.478664 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.478629 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3f6f362-14ed-4309-8041-6dc8105fa690-maas-api-tls\") pod \"maas-api-595cc648c-fj5sh\" (UID: \"a3f6f362-14ed-4309-8041-6dc8105fa690\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" Oct 29 23:47:19.478814 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.478685 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3f6f362-14ed-4309-8041-6dc8105fa690-openshift-service-ca\") pod \"maas-api-595cc648c-fj5sh\" (UID: \"a3f6f362-14ed-4309-8041-6dc8105fa690\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" Oct 29 23:47:19.478814 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.478758 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3f6f362-14ed-4309-8041-6dc8105fa690-metrics-certs\") pod \"maas-api-595cc648c-fj5sh\" (UID: \"a3f6f362-14ed-4309-8041-6dc8105fa690\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" Oct 29 23:47:19.478894 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.478873 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg9l7\" (UniqueName: \"kubernetes.io/projected/a3f6f362-14ed-4309-8041-6dc8105fa690-kube-api-access-fg9l7\") pod \"maas-api-595cc648c-fj5sh\" (UID: \"a3f6f362-14ed-4309-8041-6dc8105fa690\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" Oct 29 23:47:19.518216 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.518188 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-k7tlz\"" Oct 29 23:47:19.580353 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.580316 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3f6f362-14ed-4309-8041-6dc8105fa690-maas-api-tls\") pod \"maas-api-595cc648c-fj5sh\" (UID: \"a3f6f362-14ed-4309-8041-6dc8105fa690\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" Oct 29 23:47:19.580353 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.580353 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3f6f362-14ed-4309-8041-6dc8105fa690-openshift-service-ca\") pod \"maas-api-595cc648c-fj5sh\" (UID: \"a3f6f362-14ed-4309-8041-6dc8105fa690\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" Oct 29 23:47:19.580566 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.580369 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3f6f362-14ed-4309-8041-6dc8105fa690-metrics-certs\") pod \"maas-api-595cc648c-fj5sh\" (UID: \"a3f6f362-14ed-4309-8041-6dc8105fa690\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" Oct 29 23:47:19.580566 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.580398 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg9l7\" (UniqueName: \"kubernetes.io/projected/a3f6f362-14ed-4309-8041-6dc8105fa690-kube-api-access-fg9l7\") pod \"maas-api-595cc648c-fj5sh\" (UID: \"a3f6f362-14ed-4309-8041-6dc8105fa690\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" Oct 29 23:47:19.581019 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.580994 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a3f6f362-14ed-4309-8041-6dc8105fa690-openshift-service-ca\") pod \"maas-api-595cc648c-fj5sh\" (UID: \"a3f6f362-14ed-4309-8041-6dc8105fa690\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" Oct 29 23:47:19.582865 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.582846 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a3f6f362-14ed-4309-8041-6dc8105fa690-maas-api-tls\") pod \"maas-api-595cc648c-fj5sh\" (UID: \"a3f6f362-14ed-4309-8041-6dc8105fa690\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" Oct 29 23:47:19.582954 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.582883 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a3f6f362-14ed-4309-8041-6dc8105fa690-metrics-certs\") pod \"maas-api-595cc648c-fj5sh\" (UID: \"a3f6f362-14ed-4309-8041-6dc8105fa690\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" Oct 29 23:47:19.630390 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.630366 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg9l7\" (UniqueName: \"kubernetes.io/projected/a3f6f362-14ed-4309-8041-6dc8105fa690-kube-api-access-fg9l7\") pod \"maas-api-595cc648c-fj5sh\" (UID: \"a3f6f362-14ed-4309-8041-6dc8105fa690\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" Oct 29 23:47:19.730793 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:19.730715 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" Oct 29 23:47:20.031328 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:20.031305 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-fj5sh"] Oct 29 23:47:20.033552 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:47:20.033524 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3f6f362_14ed_4309_8041_6dc8105fa690.slice/crio-3a1a2c0bfd7abd4c66f22e9731d6bb86f9c47ca9c462cfc40f9154eec13e8d51 WatchSource:0}: Error finding container 3a1a2c0bfd7abd4c66f22e9731d6bb86f9c47ca9c462cfc40f9154eec13e8d51: Status 404 returned error can't find the container with id 3a1a2c0bfd7abd4c66f22e9731d6bb86f9c47ca9c462cfc40f9154eec13e8d51 Oct 29 23:47:20.034934 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:20.034918 5056 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 29 23:47:20.363413 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:20.363329 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" event={"ID":"a3f6f362-14ed-4309-8041-6dc8105fa690","Type":"ContainerStarted","Data":"3a1a2c0bfd7abd4c66f22e9731d6bb86f9c47ca9c462cfc40f9154eec13e8d51"} Oct 29 23:47:21.369982 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:21.369943 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" event={"ID":"a3f6f362-14ed-4309-8041-6dc8105fa690","Type":"ContainerStarted","Data":"a6c46349f2421c74aeb7da2dde8ac42da0966c7b91ea7791743011152f67856c"} Oct 29 23:47:21.370475 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:21.370023 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" Oct 29 23:47:21.427407 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:21.427354 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" podStartSLOduration=2.057888486 podStartE2EDuration="2.427340056s" totalImagesPullingTime="369.45157ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-29 23:47:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:47:21.426160899 +0000 UTC m=+2068.792908529" watchObservedRunningTime="2026-10-29 23:47:21.427340056 +0000 UTC m=+2068.794087686" Oct 29 23:47:22.415873 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:47:22.415837 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:47:27.380734 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:27.380704 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-fj5sh" Oct 29 23:47:27.457590 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:27.457548 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-dg55x"] Oct 29 23:47:27.457813 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:27.457761 5056 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" podUID="95874d50-ec6c-4370-919a-47568d8e8834" containerName="maas-api" containerID="cri-o://86db50973e1d184bd61edf3ecf9ff686c1bec64e753604ae922cc033a9939e7d" gracePeriod=30 Oct 29 23:47:28.821054 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:28.820984 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:47:28.893060 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:28.891182 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95874d50-ec6c-4370-919a-47568d8e8834-kube-api-access-kdzmq\" (UniqueName: \"kubernetes.io/projected/95874d50-ec6c-4370-919a-47568d8e8834-kube-api-access-kdzmq\") pod \"95874d50-ec6c-4370-919a-47568d8e8834\" (UID: \"95874d50-ec6c-4370-919a-47568d8e8834\") " Oct 29 23:47:28.893060 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:28.891373 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95874d50-ec6c-4370-919a-47568d8e8834-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95874d50-ec6c-4370-919a-47568d8e8834-openshift-service-ca\") pod \"95874d50-ec6c-4370-919a-47568d8e8834\" (UID: \"95874d50-ec6c-4370-919a-47568d8e8834\") " Oct 29 23:47:28.893060 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:28.891430 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95874d50-ec6c-4370-919a-47568d8e8834-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95874d50-ec6c-4370-919a-47568d8e8834-maas-api-tls\") pod \"95874d50-ec6c-4370-919a-47568d8e8834\" (UID: \"95874d50-ec6c-4370-919a-47568d8e8834\") " Oct 29 23:47:28.893060 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:28.891481 5056 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95874d50-ec6c-4370-919a-47568d8e8834-metrics-certs\" (UniqueName: \"kubernetes.io/secret/95874d50-ec6c-4370-919a-47568d8e8834-metrics-certs\") pod \"95874d50-ec6c-4370-919a-47568d8e8834\" (UID: \"95874d50-ec6c-4370-919a-47568d8e8834\") " Oct 29 23:47:28.900906 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:28.900572 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95874d50-ec6c-4370-919a-47568d8e8834-openshift-service-ca" pod "95874d50-ec6c-4370-919a-47568d8e8834" (UID: "95874d50-ec6c-4370-919a-47568d8e8834"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 29 23:47:28.904660 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:28.904556 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95874d50-ec6c-4370-919a-47568d8e8834-kube-api-access-kdzmq" pod "95874d50-ec6c-4370-919a-47568d8e8834" (UID: "95874d50-ec6c-4370-919a-47568d8e8834"). InnerVolumeSpecName "kube-api-access-kdzmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 29 23:47:28.904660 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:28.904632 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95874d50-ec6c-4370-919a-47568d8e8834-metrics-certs" pod "95874d50-ec6c-4370-919a-47568d8e8834" (UID: "95874d50-ec6c-4370-919a-47568d8e8834"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:47:28.907497 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:28.907458 5056 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95874d50-ec6c-4370-919a-47568d8e8834-maas-api-tls" pod "95874d50-ec6c-4370-919a-47568d8e8834" (UID: "95874d50-ec6c-4370-919a-47568d8e8834"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 29 23:47:28.992623 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:28.992583 5056 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdzmq\" (UniqueName: \"kubernetes.io/projected/95874d50-ec6c-4370-919a-47568d8e8834-kube-api-access-kdzmq\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:47:28.992623 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:28.992620 5056 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95874d50-ec6c-4370-919a-47568d8e8834-openshift-service-ca\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:47:28.992876 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:28.992636 5056 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95874d50-ec6c-4370-919a-47568d8e8834-maas-api-tls\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:47:28.992876 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:28.992651 5056 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95874d50-ec6c-4370-919a-47568d8e8834-metrics-certs\") on node \"ip-10-0-130-86.ec2.internal\" DevicePath \"\"" Oct 29 23:47:29.470285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:29.469766 5056 generic.go:356] "Generic (PLEG): container finished" podID="95874d50-ec6c-4370-919a-47568d8e8834" containerID="86db50973e1d184bd61edf3ecf9ff686c1bec64e753604ae922cc033a9939e7d" exitCode=0 Oct 29 23:47:29.470285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:29.469903 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" event={"ID":"95874d50-ec6c-4370-919a-47568d8e8834","Type":"ContainerDied","Data":"86db50973e1d184bd61edf3ecf9ff686c1bec64e753604ae922cc033a9939e7d"} Oct 29 23:47:29.470285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:29.469952 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" event={"ID":"95874d50-ec6c-4370-919a-47568d8e8834","Type":"ContainerDied","Data":"aa4255a1b9fb5968ff4d79c87a3acbf60b0d67ad5cba63c023742cf6d84e5bc8"} Oct 29 23:47:29.470285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:29.469981 5056 scope.go:122] "RemoveContainer" containerID="86db50973e1d184bd61edf3ecf9ff686c1bec64e753604ae922cc033a9939e7d" Oct 29 23:47:29.470285 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:29.470124 5056 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-dg55x" Oct 29 23:47:29.557799 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:29.557419 5056 scope.go:122] "RemoveContainer" containerID="86db50973e1d184bd61edf3ecf9ff686c1bec64e753604ae922cc033a9939e7d" Oct 29 23:47:29.574313 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:47:29.558147 5056 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86db50973e1d184bd61edf3ecf9ff686c1bec64e753604ae922cc033a9939e7d\": container with ID starting with 86db50973e1d184bd61edf3ecf9ff686c1bec64e753604ae922cc033a9939e7d not found: ID does not exist" containerID="86db50973e1d184bd61edf3ecf9ff686c1bec64e753604ae922cc033a9939e7d" Oct 29 23:47:29.574313 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:29.558190 5056 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86db50973e1d184bd61edf3ecf9ff686c1bec64e753604ae922cc033a9939e7d"} err="failed to get container status \"86db50973e1d184bd61edf3ecf9ff686c1bec64e753604ae922cc033a9939e7d\": rpc error: code = NotFound desc = could not find container \"86db50973e1d184bd61edf3ecf9ff686c1bec64e753604ae922cc033a9939e7d\": container with ID starting with 86db50973e1d184bd61edf3ecf9ff686c1bec64e753604ae922cc033a9939e7d not found: ID does not exist" Oct 29 23:47:29.597287 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:29.575551 5056 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-dg55x"] Oct 29 23:47:29.597561 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:29.597492 5056 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-dg55x"] Oct 29 23:47:30.802805 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:30.802761 5056 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95874d50-ec6c-4370-919a-47568d8e8834" path="/var/lib/kubelet/pods/95874d50-ec6c-4370-919a-47568d8e8834/volumes" Oct 29 23:47:47.341208 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:47.341175 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:47:52.428303 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:47:52.428237 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:47:53.118294 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:53.118235 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:47:53.119385 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:47:53.119361 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:48:19.414343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:19.413320 5056 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-746ff8f49f-9k9s2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 29 23:48:19.414343 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:19.413400 5056 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-746ff8f49f-9k9s2" podUID="a27239f9-531e-4f44-be00-22c11b243a3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 29 23:48:22.415541 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:48:22.415503 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:48:29.657973 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:29.657940 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-95ff59779-qx97f_2872686b-a6b2-4f41-9c67-c3a095753570/manager/0.log" Oct 29 23:48:30.286714 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:30.286682 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-observability-584f99c788-bbcsr_9607303a-926f-43a9-b17b-c7666e93321e/manager/0.log" Oct 29 23:48:30.789221 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:30.789190 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57666cc899-7msc4_b79d1232-88bc-4d1d-8a7c-78cc858b36f5/copy-manifests/0.log" Oct 29 23:48:30.822116 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:30.822088 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57666cc899-7msc4_b79d1232-88bc-4d1d-8a7c-78cc858b36f5/manager/0.log" Oct 29 23:48:31.246509 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:31.246476 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-phwkt_7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23/proxy/0.log" Oct 29 23:48:32.088465 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:32.088433 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-fj5sh_a3f6f362-14ed-4309-8041-6dc8105fa690/maas-api/0.log" Oct 29 23:48:34.122700 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:34.122647 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-xvxk7_067da039-3d5e-40f2-b2bf-a7ec57feed06/postgres/0.log" Oct 29 23:48:35.932103 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:35.932073 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d6dd76dc-pcg45_632b1b15-84f4-4886-a72e-6890e4f0ac19/manager/0.log" Oct 29 23:48:36.512589 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:36.512558 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-snhbq_2ca7b491-54eb-444a-a35d-3ff844706209/manager/0.log" Oct 29 23:48:37.946788 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:37.946759 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-2dl5p_712e0b41-1ea6-44f3-8021-f619394ecd34/discovery/0.log" Oct 29 23:48:38.345778 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:38.345693 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8578b44655-zxqwh_bca57086-0196-48e5-ace2-92dbfa7bb5f6/kube-auth-proxy/0.log" Oct 29 23:48:38.741494 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:38.741466 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-9k9s2_a27239f9-531e-4f44-be00-22c11b243a3a/istio-proxy/0.log" Oct 29 23:48:38.948443 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:38.948407 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-blm5t_d81526c4-5f8d-4833-beac-6cc8507caba9/istio-proxy/0.log" Oct 29 23:48:39.513768 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:39.513742 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-mj46x_a114874c-df13-4ea7-8518-c5d8725a1cb2/istio-proxy/0.log" Oct 29 23:48:39.723525 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:39.723495 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-nbfzf_5adc898f-40d4-4851-af95-1f400d519360/istio-proxy/0.log" Oct 29 23:48:40.535073 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:40.535042 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-4xjsb_27a25029-6749-4ba3-a00e-16e7bb16981b/payload-pre-processing/0.log" Oct 29 23:48:40.734033 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:40.734004 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-rfjdl_1064c092-6e9b-4ea9-a4de-4a6961a528a5/payload-processing/0.log" Oct 29 23:48:40.930737 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:40.930705 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-866787b6f9-24krt_b6ad3759-e11c-476d-82ee-fffdf8fd6d47/router/0.log" Oct 29 23:48:41.890637 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:41.890608 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp_ee561e87-9076-42d1-902f-12dc84af0988/main/0.log" Oct 29 23:48:42.290948 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:42.290876 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-xwh8q_ae66d7aa-6da8-48c2-992c-73bc167130a1/main/0.log" Oct 29 23:48:42.476179 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:42.476150 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-7pbft_36624910-931c-4b68-a211-ec47fa0beebc/main/0.log" Oct 29 23:48:47.464888 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:47.464831 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:48:52.415230 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:48:52.415190 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:48:54.253677 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:54.253650 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-95ff59779-qx97f_2872686b-a6b2-4f41-9c67-c3a095753570/manager/0.log" Oct 29 23:48:54.876841 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:54.876811 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-observability-584f99c788-bbcsr_9607303a-926f-43a9-b17b-c7666e93321e/manager/0.log" Oct 29 23:48:55.381669 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:55.381641 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57666cc899-7msc4_b79d1232-88bc-4d1d-8a7c-78cc858b36f5/copy-manifests/0.log" Oct 29 23:48:55.411147 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:55.411124 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57666cc899-7msc4_b79d1232-88bc-4d1d-8a7c-78cc858b36f5/manager/0.log" Oct 29 23:48:55.837890 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:55.837854 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-phwkt_7da3a0b7-1f7f-4b3e-a8f5-cf031895ef23/proxy/0.log" Oct 29 23:48:56.374868 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:56.374821 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-fj5sh_a3f6f362-14ed-4309-8041-6dc8105fa690/maas-api/0.log" Oct 29 23:48:58.437008 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:58.436978 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-xvxk7_067da039-3d5e-40f2-b2bf-a7ec57feed06/postgres/0.log" Oct 29 23:48:59.836918 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:48:59.836890 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d6dd76dc-pcg45_632b1b15-84f4-4886-a72e-6890e4f0ac19/manager/0.log" Oct 29 23:49:00.408223 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:00.408193 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-snhbq_2ca7b491-54eb-444a-a35d-3ff844706209/manager/0.log" Oct 29 23:49:01.616628 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:01.616599 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-2dl5p_712e0b41-1ea6-44f3-8021-f619394ecd34/discovery/0.log" Oct 29 23:49:01.991043 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:01.991007 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8578b44655-zxqwh_bca57086-0196-48e5-ace2-92dbfa7bb5f6/kube-auth-proxy/0.log" Oct 29 23:49:02.383155 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:02.383065 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-9k9s2_a27239f9-531e-4f44-be00-22c11b243a3a/istio-proxy/0.log" Oct 29 23:49:02.574718 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:02.574683 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-blm5t_d81526c4-5f8d-4833-beac-6cc8507caba9/istio-proxy/0.log" Oct 29 23:49:03.131999 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:03.131962 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-mj46x_a114874c-df13-4ea7-8518-c5d8725a1cb2/istio-proxy/0.log" Oct 29 23:49:03.317481 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:03.317447 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-nbfzf_5adc898f-40d4-4851-af95-1f400d519360/istio-proxy/0.log" Oct 29 23:49:04.088774 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:04.088746 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-4xjsb_27a25029-6749-4ba3-a00e-16e7bb16981b/payload-pre-processing/0.log" Oct 29 23:49:04.284437 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:04.284408 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-rfjdl_1064c092-6e9b-4ea9-a4de-4a6961a528a5/payload-processing/0.log" Oct 29 23:49:04.471611 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:04.471586 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-866787b6f9-24krt_b6ad3759-e11c-476d-82ee-fffdf8fd6d47/router/0.log" Oct 29 23:49:05.025336 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:05.025304 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-wkzdp_ee561e87-9076-42d1-902f-12dc84af0988/main/0.log" Oct 29 23:49:05.387902 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:05.387828 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-xwh8q_ae66d7aa-6da8-48c2-992c-73bc167130a1/main/0.log" Oct 29 23:49:05.567326 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:05.567292 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-7pbft_36624910-931c-4b68-a211-ec47fa0beebc/main/0.log" Oct 29 23:49:22.414790 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:49:22.414751 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:49:23.415949 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:23.415902 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:49:39.043372 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:39.043287 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:49:41.007119 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:41.007085 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-95ff59779-qx97f_2872686b-a6b2-4f41-9c67-c3a095753570/manager/0.log" Oct 29 23:49:41.306084 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:41.306001 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-95ff59779-qx97f_2872686b-a6b2-4f41-9c67-c3a095753570/manager/0.log" Oct 29 23:49:42.052450 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:42.052421 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-rfjdl_1064c092-6e9b-4ea9-a4de-4a6961a528a5/payload-processing/0.log" Oct 29 23:49:42.232883 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:42.232855 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-rfjdl_1064c092-6e9b-4ea9-a4de-4a6961a528a5/payload-processing/0.log" Oct 29 23:49:47.574279 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:49:47.574233 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:49:52.415253 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:49:52.415214 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:50:22.415050 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:50:22.415015 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:50:25.248873 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:25.248843 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-95ff59779-qx97f_2872686b-a6b2-4f41-9c67-c3a095753570/manager/0.log" Oct 29 23:50:25.545204 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:25.545122 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-95ff59779-qx97f_2872686b-a6b2-4f41-9c67-c3a095753570/manager/0.log" Oct 29 23:50:26.242328 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:26.242299 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-rfjdl_1064c092-6e9b-4ea9-a4de-4a6961a528a5/payload-processing/0.log" Oct 29 23:50:26.433309 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:26.433282 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-rfjdl_1064c092-6e9b-4ea9-a4de-4a6961a528a5/payload-processing/0.log" Oct 29 23:50:26.850797 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:26.850760 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:50:27.053742 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:27.053710 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:50:30.908712 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:30.908684 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gz4mg_cf41b924-24b7-4a89-9d11-af8b4246fcde/global-pull-secret-syncer/0.log" Oct 29 23:50:31.463040 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:31.463013 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-74vqh_bf94acd7-605c-4d9a-9021-5ce66a126edd/connection-checker/0.log" Oct 29 23:50:31.585552 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:31.585519 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-mjz4s_ae184786-7bc6-403a-87fc-185baf65ffa5/connection-checker/0.log" Oct 29 23:50:31.704949 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:31.704923 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-z6qn8_5d039e77-d1dc-4215-9dd0-2a95e6d7e8fd/connection-checker/0.log" Oct 29 23:50:31.843841 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:31.843772 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" Oct 29 23:50:32.293241 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:32.293213 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-86.ec2.internal_47e5a52938e8a36be364af182a627291/haproxy/0.log" Oct 29 23:50:36.766107 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:36.766069 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:50:36.917998 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:36.917965 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:50:38.828830 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:38.828800 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d6dd76dc-pcg45_632b1b15-84f4-4886-a72e-6890e4f0ac19/manager/0.log" Oct 29 23:50:39.250521 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:39.250492 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-57bb9bfff-snhbq_2ca7b491-54eb-444a-a35d-3ff844706209/manager/0.log" Oct 29 23:50:40.747417 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.747382 5056 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx"] Oct 29 23:50:40.747860 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.747846 5056 cpu_manager.go:414] "containerMap: removing container" podUID="95874d50-ec6c-4370-919a-47568d8e8834" containerName="maas-api" Oct 29 23:50:40.747903 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.747861 5056 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95874d50-ec6c-4370-919a-47568d8e8834" containerName="maas-api" Oct 29 23:50:40.747979 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.747969 5056 memory_manager.go:365] "RemoveStaleState removing state" podUID="95874d50-ec6c-4370-919a-47568d8e8834" containerName="maas-api" Oct 29 23:50:40.751161 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.751144 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:40.767223 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.767204 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w8hzv\"/\"openshift-service-ca.crt\"" Oct 29 23:50:40.767390 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.767374 5056 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w8hzv\"/\"kube-root-ca.crt\"" Oct 29 23:50:40.782572 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.782548 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx"] Oct 29 23:50:40.783806 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.783790 5056 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-w8hzv\"/\"default-dockercfg-l6v7j\"" Oct 29 23:50:40.795762 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.795733 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7aec6a03-8a50-4273-9307-c9f659168805-podres\") pod \"perf-node-gather-daemonset-nslgx\" (UID: \"7aec6a03-8a50-4273-9307-c9f659168805\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:40.795841 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.795780 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7aec6a03-8a50-4273-9307-c9f659168805-lib-modules\") pod \"perf-node-gather-daemonset-nslgx\" (UID: \"7aec6a03-8a50-4273-9307-c9f659168805\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:40.795941 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.795922 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjbkz\" (UniqueName: \"kubernetes.io/projected/7aec6a03-8a50-4273-9307-c9f659168805-kube-api-access-mjbkz\") pod \"perf-node-gather-daemonset-nslgx\" (UID: \"7aec6a03-8a50-4273-9307-c9f659168805\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:40.795998 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.795961 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7aec6a03-8a50-4273-9307-c9f659168805-sys\") pod \"perf-node-gather-daemonset-nslgx\" (UID: \"7aec6a03-8a50-4273-9307-c9f659168805\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:40.796086 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.796068 5056 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7aec6a03-8a50-4273-9307-c9f659168805-proc\") pod \"perf-node-gather-daemonset-nslgx\" (UID: \"7aec6a03-8a50-4273-9307-c9f659168805\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:40.897103 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.897066 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7aec6a03-8a50-4273-9307-c9f659168805-proc\") pod \"perf-node-gather-daemonset-nslgx\" (UID: \"7aec6a03-8a50-4273-9307-c9f659168805\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:40.897333 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.897184 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7aec6a03-8a50-4273-9307-c9f659168805-podres\") pod \"perf-node-gather-daemonset-nslgx\" (UID: \"7aec6a03-8a50-4273-9307-c9f659168805\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:40.897333 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.897189 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7aec6a03-8a50-4273-9307-c9f659168805-proc\") pod \"perf-node-gather-daemonset-nslgx\" (UID: \"7aec6a03-8a50-4273-9307-c9f659168805\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:40.897333 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.897211 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7aec6a03-8a50-4273-9307-c9f659168805-lib-modules\") pod \"perf-node-gather-daemonset-nslgx\" (UID: \"7aec6a03-8a50-4273-9307-c9f659168805\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:40.897333 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.897252 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7aec6a03-8a50-4273-9307-c9f659168805-podres\") pod \"perf-node-gather-daemonset-nslgx\" (UID: \"7aec6a03-8a50-4273-9307-c9f659168805\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:40.897333 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.897295 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjbkz\" (UniqueName: \"kubernetes.io/projected/7aec6a03-8a50-4273-9307-c9f659168805-kube-api-access-mjbkz\") pod \"perf-node-gather-daemonset-nslgx\" (UID: \"7aec6a03-8a50-4273-9307-c9f659168805\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:40.897333 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.897323 5056 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7aec6a03-8a50-4273-9307-c9f659168805-sys\") pod \"perf-node-gather-daemonset-nslgx\" (UID: \"7aec6a03-8a50-4273-9307-c9f659168805\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:40.897333 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.897328 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7aec6a03-8a50-4273-9307-c9f659168805-lib-modules\") pod \"perf-node-gather-daemonset-nslgx\" (UID: \"7aec6a03-8a50-4273-9307-c9f659168805\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:40.897620 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.897385 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7aec6a03-8a50-4273-9307-c9f659168805-sys\") pod \"perf-node-gather-daemonset-nslgx\" (UID: \"7aec6a03-8a50-4273-9307-c9f659168805\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:40.934460 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:40.934429 5056 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjbkz\" (UniqueName: \"kubernetes.io/projected/7aec6a03-8a50-4273-9307-c9f659168805-kube-api-access-mjbkz\") pod \"perf-node-gather-daemonset-nslgx\" (UID: \"7aec6a03-8a50-4273-9307-c9f659168805\") " pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:41.064066 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:41.063990 5056 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:41.361921 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:41.361895 5056 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx"] Oct 29 23:50:41.363864 ip-10-0-130-86 kubenswrapper[5056]: W1029 23:50:41.363835 5056 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7aec6a03_8a50_4273_9307_c9f659168805.slice/crio-cd34ee09ffacb181cc4756067cba91cbafed5b7dce0860fd7cf57254fc1f0d90 WatchSource:0}: Error finding container cd34ee09ffacb181cc4756067cba91cbafed5b7dce0860fd7cf57254fc1f0d90: Status 404 returned error can't find the container with id cd34ee09ffacb181cc4756067cba91cbafed5b7dce0860fd7cf57254fc1f0d90 Oct 29 23:50:41.540743 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:41.540706 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" event={"ID":"7aec6a03-8a50-4273-9307-c9f659168805","Type":"ContainerStarted","Data":"42a2bdcdeb372340c9a1d9f3b32b147515c2723ef37671ea76f2e8e1c62fe806"} Oct 29 23:50:41.540912 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:41.540755 5056 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" event={"ID":"7aec6a03-8a50-4273-9307-c9f659168805","Type":"ContainerStarted","Data":"cd34ee09ffacb181cc4756067cba91cbafed5b7dce0860fd7cf57254fc1f0d90"} Oct 29 23:50:41.540912 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:41.540837 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:41.584232 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:41.584185 5056 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" podStartSLOduration=1.5841728019999999 podStartE2EDuration="1.584172802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-29 23:50:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-29 23:50:41.58247655 +0000 UTC m=+2268.949224182" watchObservedRunningTime="2026-10-29 23:50:41.584172802 +0000 UTC m=+2268.950920432" Oct 29 23:50:41.912051 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:41.912020 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-t6gcj_63784b03-969b-4e2a-b799-e027ad1c0183/cluster-monitoring-operator/0.log" Oct 29 23:50:43.464357 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:43.464331 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m5k4b_7a57ebca-6da5-46dc-9315-5896905b574d/node-exporter/0.log" Oct 29 23:50:43.599249 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:43.599221 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m5k4b_7a57ebca-6da5-46dc-9315-5896905b574d/kube-rbac-proxy/0.log" Oct 29 23:50:43.723678 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:43.723599 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m5k4b_7a57ebca-6da5-46dc-9315-5896905b574d/init-textfile/0.log" Oct 29 23:50:45.210523 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:45.210494 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-6p9zt_9cb5fa50-deea-4044-a64e-fa6ea83bb1c4/prometheus-operator/0.log" Oct 29 23:50:45.337075 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:45.337036 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-6p9zt_9cb5fa50-deea-4044-a64e-fa6ea83bb1c4/kube-rbac-proxy/0.log" Oct 29 23:50:45.467397 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:45.467320 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-9jt7p_2af4443b-bbec-448d-9a41-07fcc122059b/prometheus-operator-admission-webhook/0.log" Oct 29 23:50:47.556482 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:47.556459 5056 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-w8hzv/perf-node-gather-daemonset-nslgx" Oct 29 23:50:47.671542 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:47.671511 5056 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-25m4p_c964bbd5-fa82-459f-b464-d27fd1080654/konnectivity-agent/0.log" limit=1024 Oct 29 23:50:48.218339 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:48.218309 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-b9jng_b218e5d9-83c0-4976-90da-471e6b86d460/networking-console-plugin/0.log" Oct 29 23:50:49.352282 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:49.352231 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/3.log" Oct 29 23:50:49.357158 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:49.357128 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-2gzh7_947dc777-7dd0-4244-913a-5dd068e6e6a3/console-operator/4.log" Oct 29 23:50:51.837321 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:51.837280 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-pn894_350f9704-1e0e-4f41-958b-8b0b16761375/volume-data-source-validator/0.log" Oct 29 23:50:52.415737 ip-10-0-130-86 kubenswrapper[5056]: E1029 23:50:52.415700 5056 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-x975f" containerName="extractor" Oct 29 23:50:54.156163 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:54.156132 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gnwhl_8388282a-8b2c-4f5c-ab3d-ec58ea20cc93/dns/0.log" Oct 29 23:50:54.276817 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:54.276790 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gnwhl_8388282a-8b2c-4f5c-ab3d-ec58ea20cc93/kube-rbac-proxy/0.log" Oct 29 23:50:54.498729 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:54.498649 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-2dl5p_712e0b41-1ea6-44f3-8021-f619394ecd34/discovery/0.log" Oct 29 23:50:54.611060 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:54.611032 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-8578b44655-zxqwh_bca57086-0196-48e5-ace2-92dbfa7bb5f6/kube-auth-proxy/0.log" Oct 29 23:50:54.746719 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:54.746688 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-9k9s2_a27239f9-531e-4f44-be00-22c11b243a3a/istio-proxy/0.log" Oct 29 23:50:54.813797 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:54.813720 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-blm5t_d81526c4-5f8d-4833-beac-6cc8507caba9/istio-proxy/0.log" Oct 29 23:50:54.988630 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:54.988601 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-mj46x_a114874c-df13-4ea7-8518-c5d8725a1cb2/istio-proxy/0.log" Oct 29 23:50:55.052794 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:55.052763 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zr8dl_1211641d-1d2c-4de0-b993-5f1e84edc09f/dns-node-resolver/0.log" Oct 29 23:50:55.055272 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:55.055239 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-746ff8f49f-nbfzf_5adc898f-40d4-4851-af95-1f400d519360/istio-proxy/0.log" Oct 29 23:50:55.342608 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:55.342576 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-4xjsb_27a25029-6749-4ba3-a00e-16e7bb16981b/payload-pre-processing/0.log" Oct 29 23:50:55.409151 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:55.409117 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5f78687d5b-rfjdl_1064c092-6e9b-4ea9-a4de-4a6961a528a5/payload-processing/0.log" Oct 29 23:50:55.480193 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:55.480147 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-866787b6f9-24krt_b6ad3759-e11c-476d-82ee-fffdf8fd6d47/router/0.log" Oct 29 23:50:56.375692 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:56.375664 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5768db7d69-j4qs9_e91b184c-b290-4c7e-a763-52c80c34a17d/registry/0.log" Oct 29 23:50:56.498586 ip-10-0-130-86 kubenswrapper[5056]: I1029 23:50:56.498561 5056 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jg249_c652cee1-24f4-460a-8e1a-669b0ae9c0b1/node-ca/0.log"