Nov 04 07:14:52.795783 ip-10-0-131-6 systemd[1]: Starting Kubernetes Kubelet... Nov 04 07:14:53.199995 ip-10-0-131-6 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 07:14:53.199995 ip-10-0-131-6 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 07:14:53.199995 ip-10-0-131-6 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 07:14:53.199995 ip-10-0-131-6 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 07:14:53.204721 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204642 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 07:14:53.204721 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204720 2583 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 07:14:53.204721 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204724 2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204728 2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204732 2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204736 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204739 2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204742 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204745 2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204748 2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204750 2583 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204753 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204756 2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204759 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204762 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204764 2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204767 2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204770 2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204772 2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204775 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204777 2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204780 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 07:14:53.204814 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204783 2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204786 2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204788 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204791 2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204793 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204796 2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204799 2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204801 2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204813 2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204817 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204819 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204822 2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204825 2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204828 2583 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204831 2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204833 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204836 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204839 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204841 2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204844 2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 07:14:53.205268 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204846 2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204849 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204851 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204854 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204857 2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204859 2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204862 2583 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204864 2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204868 2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204872 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204874 2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204877 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204880 2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204882 2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204886 2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204889 2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204892 2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204894 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204902 2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 07:14:53.205748 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204905 2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204908 2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204911 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204915 2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204918 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204921 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204924 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204927 2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204930 2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204933 2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204935 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204938 2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204940 2583 feature_gate.go:386] unrecognized feature gate: Example Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204943 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204946 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204949 2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204951 2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204954 2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204956 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 07:14:53.206195 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204959 2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204962 2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204965 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204968 2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204970 2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204973 2583 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204975 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204978 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204981 2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204984 2583 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204987 2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204989 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204992 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204994 2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.204997 2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.205000 2583 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.205002 2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.205005 2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.205007 2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.205010 2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 07:14:53.206655 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.205013 2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.205015 2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.205018 2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.205020 2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.205024 2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.205027 2583 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206462 2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206471 2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206475 2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206478 2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206482 2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206485 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206488 2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206491 2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206495 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206498 2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206500 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206518 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206521 2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 07:14:53.207140 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206524 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206528 2583 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206531 2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206534 2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206536 2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206539 2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206542 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206544 2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206548 2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206550 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206553 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206556 2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206558 2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206561 2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206563 2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206566 2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206570 2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206574 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 07:14:53.207649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206577 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206579 2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206582 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206584 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206588 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206591 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206593 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206596 2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206599 2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206601 2583 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206604 2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206607 2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206610 2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206612 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206615 2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206618 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206621 2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206624 2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206627 2583 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206630 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 07:14:53.208082 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206632 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206635 2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206637 2583 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206640 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206643 2583 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206645 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206648 2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206650 2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206653 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206656 2583 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206659 2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206662 2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206664 2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206667 2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206670 2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206672 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206675 2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206678 2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206680 2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206684 2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 07:14:53.208578 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206687 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206690 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206692 2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206696 2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206698 2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206701 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206703 2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206706 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206709 2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206711 2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206714 2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206717 2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206719 2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206723 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206725 2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206728 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206730 2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206733 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206736 2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 07:14:53.209054 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206738 2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206741 2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206744 2583 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206747 2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206749 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206752 2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206754 2583 feature_gate.go:386] unrecognized feature gate: Example Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206757 2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206759 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206762 2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206765 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206768 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206771 2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206774 2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206776 2583 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.206779 2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.207979 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.207987 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.207991 2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.207994 2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 07:14:53.209521 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.207997 2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208000 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208002 2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208005 2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208008 2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208010 2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208013 2583 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208016 2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208019 2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208022 2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208024 2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208027 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208029 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208032 2583 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208035 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208037 2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208040 2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208042 2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208046 2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208050 2583 feature_gate.go:386] unrecognized feature gate: Example Nov 04 07:14:53.210004 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208053 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208056 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208058 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208061 2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208063 2583 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208066 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208069 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208072 2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208074 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208077 2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208080 2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208083 2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208086 2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208089 2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208091 2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208094 2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208097 2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208100 2583 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208103 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 07:14:53.210483 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208105 2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208108 2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208110 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208113 2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208115 2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208118 2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208120 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208123 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208126 2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208128 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208131 2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208134 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208136 2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208139 2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208141 2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208144 2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208146 2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208149 2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208152 2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 07:14:53.210962 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208155 2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208158 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208161 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208163 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208172 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208175 2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208178 2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208181 2583 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208183 2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208186 2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208189 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208191 2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208194 2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208196 2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208199 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208202 2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208204 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208207 2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208209 2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208212 2583 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 07:14:53.211409 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208215 2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208217 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208220 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208222 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208225 2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208228 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208230 2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208233 2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208235 2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208238 2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208240 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208243 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208245 2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208248 2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208251 2583 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208253 2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208256 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208259 2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208262 2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 07:14:53.211896 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208264 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208267 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208270 2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208272 2583 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.208277 2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.208936 2583 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.208946 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.208953 2583 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.208958 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.208962 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.208965 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.208969 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.208975 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.208978 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.208981 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.208985 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.208988 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.208991 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.208994 2583 flags.go:64] FLAG: --cgroup-root="" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.208997 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209000 2583 flags.go:64] FLAG: --client-ca-file="" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209003 2583 flags.go:64] FLAG: --cloud-provider="external" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209006 2583 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 07:14:53.212460 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209011 2583 flags.go:64] FLAG: --cluster-domain="" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209013 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209017 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209020 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209023 2583 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209027 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209030 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209033 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209037 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209041 2583 flags.go:64] FLAG: --contention-profiling="false" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209044 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209047 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209050 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209053 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209060 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209064 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209067 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209069 2583 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209073 2583 flags.go:64] FLAG: --enable-server="true" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209076 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209080 2583 flags.go:64] FLAG: --event-burst="100" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209083 2583 flags.go:64] FLAG: --event-qps="50" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209087 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209090 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 07:14:53.213022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209093 2583 flags.go:64] FLAG: --eviction-hard="" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209098 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209101 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209104 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209107 2583 flags.go:64] FLAG: --eviction-soft="" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209110 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209113 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209116 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209119 2583 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209122 2583 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209125 2583 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209130 2583 flags.go:64] FLAG: --feature-gates="" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209134 2583 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209138 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209141 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209144 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209147 2583 flags.go:64] FLAG: --healthz-port="10248" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209152 2583 flags.go:64] FLAG: --help="false" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209155 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209158 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209161 2583 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209164 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209167 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 07:14:53.213649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209170 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209173 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209176 2583 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209179 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209182 2583 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209185 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209188 2583 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209191 2583 flags.go:64] FLAG: --kube-reserved="" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209204 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209208 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209211 2583 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209214 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209218 2583 flags.go:64] FLAG: --lock-file="" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209220 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209223 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209227 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209232 2583 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209235 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209238 2583 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209241 2583 flags.go:64] FLAG: --logging-format="text" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209246 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209249 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209252 2583 flags.go:64] FLAG: --manifest-url="" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209255 2583 flags.go:64] FLAG: --manifest-url-header="" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209259 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 07:14:53.214189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209263 2583 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209266 2583 flags.go:64] FLAG: --max-pods="110" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209270 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209273 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209276 2583 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209279 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209282 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209296 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209302 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209311 2583 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209314 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209317 2583 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209320 2583 flags.go:64] FLAG: --pod-cidr="" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209323 2583 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209326 2583 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209329 2583 flags.go:64] FLAG: --pods-per-core="0" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209332 2583 flags.go:64] FLAG: --port="10250" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209335 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209338 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-00844c6faf71f481b" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209341 2583 flags.go:64] FLAG: --qos-reserved="" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209344 2583 flags.go:64] FLAG: --read-only-port="10255" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209348 2583 flags.go:64] FLAG: --register-node="true" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209350 2583 flags.go:64] FLAG: --register-with-taints="" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209354 2583 flags.go:64] FLAG: --registry-burst="10" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209357 2583 flags.go:64] FLAG: --registry-qps="5" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209360 2583 flags.go:64] FLAG: --reserved-cpus="" Nov 04 07:14:53.214799 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209363 2583 flags.go:64] FLAG: --reserved-memory="" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209368 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209372 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209375 2583 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209378 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209381 2583 flags.go:64] FLAG: --runonce="false" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209384 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209388 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209391 2583 flags.go:64] FLAG: --seccomp-default="false" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209394 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209397 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209400 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209403 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209406 2583 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209409 2583 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209412 2583 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209415 2583 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209418 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209421 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209424 2583 flags.go:64] FLAG: --system-cgroups="" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209427 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209432 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209437 2583 flags.go:64] FLAG: --tls-cert-file="" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209440 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209445 2583 flags.go:64] FLAG: --tls-min-version="" Nov 04 07:14:53.215430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209448 2583 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209451 2583 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209454 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209457 2583 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209460 2583 flags.go:64] FLAG: --v="2" Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209464 2583 flags.go:64] FLAG: --version="false" Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209468 2583 flags.go:64] FLAG: --vmodule="" Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209472 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209477 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209622 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209626 2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209629 2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209633 2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209636 2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209639 2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209643 2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209646 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209650 2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209653 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209656 2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209658 2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 07:14:53.216061 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209661 2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209664 2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209666 2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209669 2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209672 2583 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209674 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209677 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209680 2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209684 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209687 2583 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209690 2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209692 2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209695 2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209697 2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209700 2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209703 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209706 2583 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209709 2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209712 2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209715 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 07:14:53.216610 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209718 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209721 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209723 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209726 2583 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209728 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209731 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209734 2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209737 2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209739 2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209742 2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209745 2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209748 2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209750 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209753 2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209755 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209758 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209761 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209764 2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209766 2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209769 2583 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 07:14:53.217097 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209774 2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209777 2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209780 2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209782 2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209785 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209788 2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209790 2583 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209793 2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209796 2583 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209798 2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209801 2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209804 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209807 2583 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209810 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209813 2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209815 2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209818 2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209820 2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209823 2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209826 2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 07:14:53.217588 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209828 2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209831 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209834 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209837 2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209839 2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209842 2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209844 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209847 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209849 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209852 2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209855 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209857 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209861 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209863 2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209866 2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209869 2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209871 2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209874 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209876 2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 07:14:53.218057 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209879 2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 07:14:53.218528 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209882 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 07:14:53.218528 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209884 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 07:14:53.218528 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209887 2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 07:14:53.218528 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209891 2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 07:14:53.218528 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209894 2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 07:14:53.218528 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209897 2583 feature_gate.go:386] unrecognized feature gate: Example Nov 04 07:14:53.218528 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209901 2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 07:14:53.218528 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209904 2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 07:14:53.218528 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209907 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 07:14:53.218528 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209910 2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 07:14:53.218528 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209913 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 07:14:53.218528 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209916 2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 07:14:53.218528 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209919 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 07:14:53.218528 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209921 2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 07:14:53.218528 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.209947 2583 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209983 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209987 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209990 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209992 2583 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209995 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.209998 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210000 2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210003 2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210006 2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210009 2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210013 2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210016 2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210018 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210021 2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210023 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210026 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210029 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210032 2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210035 2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 07:14:53.218908 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210037 2583 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210042 2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210044 2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210047 2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210049 2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210052 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210055 2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210057 2583 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210060 2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210062 2583 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210065 2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210068 2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210070 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210073 2583 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210076 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210079 2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210081 2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210084 2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210087 2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210089 2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 07:14:53.219373 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210092 2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210094 2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210097 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210101 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210103 2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210106 2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210109 2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210112 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210114 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210117 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210120 2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210123 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210125 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210129 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210132 2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210134 2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210137 2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210140 2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210142 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 07:14:53.219870 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210145 2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210147 2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210150 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210152 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210155 2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210158 2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210160 2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210163 2583 feature_gate.go:386] unrecognized feature gate: Example Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210168 2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210172 2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210175 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210178 2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210181 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210184 2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210186 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210189 2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210193 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210195 2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210198 2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 07:14:53.220314 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210200 2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210203 2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210206 2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210210 2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210213 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210216 2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210219 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210223 2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210226 2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210228 2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210231 2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210233 2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210236 2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210239 2583 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210241 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210244 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210246 2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210249 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210251 2583 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 07:14:53.220771 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210254 2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210256 2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210259 2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210262 2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210265 2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210268 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210271 2583 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210273 2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210276 2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.210279 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.217635 2583 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.217651 2583 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.217665 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.217670 2583 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217724 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217730 2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217734 2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217737 2583 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217740 2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217743 2583 feature_gate.go:386] unrecognized feature gate: Example Nov 04 07:14:53.221292 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217746 2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217748 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217751 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217754 2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217757 2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217759 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217762 2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217765 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217768 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217771 2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217773 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217776 2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217778 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217781 2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217784 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217786 2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217789 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217792 2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217794 2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 07:14:53.221808 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217797 2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217800 2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217803 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217805 2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217810 2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217812 2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217815 2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217818 2583 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217820 2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217823 2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217826 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217828 2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217831 2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217834 2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217836 2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217839 2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217842 2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217844 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217847 2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217849 2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 07:14:53.222302 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217852 2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217855 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217857 2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217860 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217862 2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217865 2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217868 2583 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217870 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217873 2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217875 2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217878 2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217880 2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217883 2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217885 2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217888 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217892 2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217894 2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217898 2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217901 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217903 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 07:14:53.222880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217906 2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217908 2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217911 2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217914 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217916 2583 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217919 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217921 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217925 2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217929 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217932 2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217935 2583 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217938 2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217940 2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217943 2583 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217946 2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217948 2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217951 2583 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217953 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217956 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217959 2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 07:14:53.223354 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217961 2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217964 2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217967 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217969 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217972 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217975 2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217980 2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217983 2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217986 2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217989 2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217992 2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217995 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.217997 2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218000 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218003 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218005 2583 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218008 2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218011 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218013 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 07:14:53.224256 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218016 2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 07:14:53.225000 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218018 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 07:14:53.225000 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.218047 2583 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 07:14:53.225000 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218182 2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 07:14:53.225000 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218187 2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 07:14:53.225000 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218190 2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 07:14:53.225000 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218192 2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 07:14:53.225000 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218195 2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 07:14:53.225000 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218198 2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 07:14:53.225000 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218200 2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 07:14:53.225000 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218203 2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 07:14:53.225000 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218206 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 07:14:53.225000 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218208 2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 07:14:53.225000 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218211 2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 07:14:53.225000 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218214 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 07:14:53.225000 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218217 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 07:14:53.225000 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218219 2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218222 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218225 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218227 2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218230 2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218233 2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218235 2583 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218238 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218241 2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218244 2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218246 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218249 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218251 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218254 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218256 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218259 2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218262 2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218264 2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218267 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 07:14:53.225466 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218269 2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218272 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218274 2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218277 2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218280 2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218282 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218285 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218288 2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218291 2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218293 2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218296 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218298 2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218301 2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218304 2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218306 2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218309 2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218311 2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218314 2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218317 2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 07:14:53.225938 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218319 2583 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218322 2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218325 2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218327 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218330 2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218333 2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218335 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218338 2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218340 2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218343 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218345 2583 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218348 2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218350 2583 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218353 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218356 2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218358 2583 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218361 2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218363 2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218366 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218369 2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 07:14:53.226412 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218371 2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218374 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218376 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218379 2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218382 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218384 2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218387 2583 feature_gate.go:386] unrecognized feature gate: Example Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218391 2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218395 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218398 2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218401 2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218404 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218407 2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218410 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218413 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218416 2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218418 2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218426 2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218429 2583 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 07:14:53.226935 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218431 2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 07:14:53.227611 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218434 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 07:14:53.227611 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218437 2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 07:14:53.227611 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218439 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 07:14:53.227611 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218442 2583 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 07:14:53.227611 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218445 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 07:14:53.227611 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218447 2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 07:14:53.227611 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218450 2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 07:14:53.227611 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218452 2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 07:14:53.227611 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218455 2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 07:14:53.227611 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218458 2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 07:14:53.227611 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218460 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 07:14:53.227611 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218463 2583 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 07:14:53.227611 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218466 2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 07:14:53.227611 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218468 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 07:14:53.227611 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218471 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 07:14:53.227611 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.218493 2583 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218542 2583 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218547 2583 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218550 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218553 2583 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218556 2583 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218558 2583 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218561 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218564 2583 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218566 2583 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218569 2583 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218572 2583 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218574 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218577 2583 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218582 2583 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218585 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218587 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218590 2583 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218592 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218595 2583 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 07:14:53.228229 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218598 2583 feature_gate.go:386] unrecognized feature gate: Example Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218602 2583 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218606 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218609 2583 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218611 2583 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218614 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218617 2583 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218619 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218622 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218624 2583 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218627 2583 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218631 2583 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218634 2583 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218637 2583 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218640 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218643 2583 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218646 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218648 2583 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218651 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 07:14:53.228779 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218654 2583 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218656 2583 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218659 2583 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218661 2583 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218664 2583 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218666 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218669 2583 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218673 2583 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218676 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218679 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218681 2583 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218684 2583 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218687 2583 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218689 2583 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218691 2583 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218694 2583 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218696 2583 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218700 2583 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218702 2583 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218705 2583 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 07:14:53.229317 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218707 2583 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218710 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218713 2583 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218716 2583 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218718 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218721 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218723 2583 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218726 2583 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218729 2583 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218731 2583 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218734 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218737 2583 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218739 2583 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218742 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218745 2583 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218748 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218750 2583 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218752 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218755 2583 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 07:14:53.229828 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218759 2583 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218761 2583 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218764 2583 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218767 2583 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218769 2583 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218772 2583 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218774 2583 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218777 2583 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218779 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218782 2583 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218785 2583 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218787 2583 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218792 2583 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218795 2583 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218797 2583 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218800 2583 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218802 2583 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218805 2583 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218807 2583 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 07:14:53.230277 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218810 2583 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 07:14:53.230915 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218812 2583 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 07:14:53.230915 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218816 2583 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 07:14:53.230915 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218819 2583 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 07:14:53.230915 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218821 2583 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 07:14:53.230915 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218824 2583 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 07:14:53.230915 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218827 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 07:14:53.230915 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218829 2583 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 07:14:53.230915 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218832 2583 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 07:14:53.230915 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:53.218834 2583 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 07:14:53.230915 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.218890 2583 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 07:14:53.230915 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.220792 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Nov 04 07:14:53.230915 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.221728 2583 server.go:1014] "Starting client certificate rotation" Nov 04 07:14:53.230915 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.221836 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 07:14:53.230915 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.222524 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 07:14:53.230915 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.224321 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 07:14:53.230915 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.227337 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 07:14:53.238043 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.238019 2583 log.go:25] "Validated CRI v1 runtime API" Nov 04 07:14:53.242049 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.242031 2583 log.go:25] "Validated CRI v1 image API" Nov 04 07:14:53.242914 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.242886 2583 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 07:14:53.246561 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.246525 2583 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ae07571e-34ea-4bf3-b8c3-f6d9767bd153:/dev/nvme0n1p4 c4f37c9e-820d-483e-b1ba-109835749dd0:/dev/nvme0n1p3] Nov 04 07:14:53.246636 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.246560 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Nov 04 07:14:53.252436 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.252309 2583 manager.go:217] Machine: {Timestamp:2026-11-04 07:14:53.250450416 +0000 UTC m=+0.339977346 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3186147 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2de77bb6519bb8ed936be41ce50486 SystemUUID:ec2de77b-b651-9bb8-ed93-6be41ce50486 BootID:9279c8cf-a8da-4075-8505-954e6c7043cd Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d1:c3:80:4b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d1:c3:80:4b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:38:a6:f2:69:01 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 07:14:53.252436 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.252429 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 07:14:53.252638 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.252606 2583 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 07:14:53.255806 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.255778 2583 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 07:14:53.255939 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.255808 2583 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-6.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 07:14:53.255982 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.255941 2583 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 07:14:53.255982 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.255949 2583 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 07:14:53.256042 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.255990 2583 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 07:14:53.256042 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.256003 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 07:14:53.256933 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.256923 2583 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 07:14:53.259280 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.259263 2583 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 07:14:53.260651 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.260639 2583 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 07:14:53.260768 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.260759 2583 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 07:14:53.260806 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.260790 2583 kubelet.go:493] "Attempting to sync node with API server" Nov 04 07:14:53.260806 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.260800 2583 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 07:14:53.260893 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.260814 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 07:14:53.260893 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.260823 2583 kubelet.go:405] "Adding apiserver pod source" Nov 04 07:14:53.260893 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.260831 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 07:14:53.261370 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.261353 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 07:14:53.264143 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.264126 2583 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 07:14:53.269300 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.269284 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 07:14:53.270168 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.270155 2583 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 07:14:53.270222 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.270185 2583 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 07:14:53.270336 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.270326 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 07:14:53.270365 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.270341 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 07:14:53.270365 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.270347 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 07:14:53.270365 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.270352 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 07:14:53.270365 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.270358 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 07:14:53.270365 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.270364 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 07:14:53.271001 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.270991 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 07:14:53.271037 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.271001 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 07:14:53.271037 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.271008 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 07:14:53.271037 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.271015 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 07:14:53.271037 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.271023 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 07:14:53.271037 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.271033 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 07:14:53.271651 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.271643 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 07:14:53.271651 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.271652 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 07:14:53.275392 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.275373 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-6.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 04 07:14:53.275494 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.275483 2583 server.go:1263] "Started kubelet" Nov 04 07:14:53.275602 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.275570 2583 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 07:14:53.275699 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.275623 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 07:14:53.275744 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.275698 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 07:14:53.276270 ip-10-0-131-6 systemd[1]: Started Kubernetes Kubelet. Nov 04 07:14:53.276845 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.276822 2583 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 07:14:53.277896 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.277884 2583 server.go:317] "Adding debug handlers to kubelet server" Nov 04 07:14:53.282286 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.282267 2583 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Nov 04 07:14:53.282718 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.282698 2583 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-6.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node" Nov 04 07:14:53.283643 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.282636 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-6.us-east-2.compute.internal.18e4c63780212a65 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-6.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-6.us-east-2.compute.internal,},FirstTimestamp:2026-11-04 07:14:53.275458149 +0000 UTC m=+0.364985080,LastTimestamp:2026-11-04 07:14:53.275458149 +0000 UTC m=+0.364985080,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-6.us-east-2.compute.internal,}" Nov 04 07:14:53.284646 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.284626 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-26qkg" Nov 04 07:14:53.284724 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.284682 2583 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 07:14:53.284724 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.284700 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 07:14:53.284841 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.284824 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" Nov 04 07:14:53.285030 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.285012 2583 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Nov 04 07:14:53.285453 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.285436 2583 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 07:14:53.285639 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.285622 2583 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 07:14:53.285639 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.285436 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 07:14:53.285639 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.285484 2583 factory.go:55] Registering systemd factory Nov 04 07:14:53.285815 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.285662 2583 factory.go:223] Registration of the systemd container factory successfully Nov 04 07:14:53.285815 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.285767 2583 reconstruct.go:97] "Volume reconstruction finished" Nov 04 07:14:53.285815 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.285776 2583 reconciler.go:29] "Reconciler: start to sync state" Nov 04 07:14:53.285935 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.285832 2583 factory.go:153] Registering CRI-O factory Nov 04 07:14:53.285935 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.285842 2583 factory.go:223] Registration of the crio container factory successfully Nov 04 07:14:53.285935 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.285906 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 07:14:53.285935 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.285933 2583 factory.go:103] Registering Raw factory Nov 04 07:14:53.286118 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.285949 2583 manager.go:1199] Started watching for new ooms in manager Nov 04 07:14:53.286925 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.286908 2583 manager.go:319] Starting recovery of all containers Nov 04 07:14:53.295300 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.295258 2583 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-6.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Nov 04 07:14:53.298802 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.298719 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-26qkg" Nov 04 07:14:53.299878 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.299860 2583 manager.go:324] Recovery completed Nov 04 07:14:53.302412 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.302391 2583 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Nov 04 07:14:53.304384 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.304365 2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 07:14:53.306086 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.306069 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 07:14:53.306176 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.306102 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 07:14:53.306176 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.306117 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 07:14:53.306734 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.306721 2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 07:14:53.306819 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.306719 2583 cpu_manager.go:226] "Starting" policy="none" Nov 04 07:14:53.306819 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.306751 2583 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 07:14:53.306819 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.306766 2583 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 07:14:53.307407 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.307392 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 07:14:53.307448 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.307422 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 07:14:53.307448 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.307435 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 07:14:53.308826 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.308814 2583 policy_none.go:50] "Start" Nov 04 07:14:53.308883 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.308829 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 07:14:53.308883 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.308839 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 07:14:53.310837 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.310826 2583 policy_none.go:44] "Start" Nov 04 07:14:53.350074 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.350050 2583 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 07:14:53.354052 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.354039 2583 manager.go:341] "Starting Device Plugin manager" Nov 04 07:14:53.354108 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.354064 2583 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 07:14:53.354108 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.354072 2583 server.go:87] "Starting device plugin registration server" Nov 04 07:14:53.354309 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.354297 2583 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 07:14:53.354352 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.354313 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 07:14:53.354555 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.354400 2583 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 07:14:53.354555 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.354474 2583 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 07:14:53.354555 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.354483 2583 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 07:14:53.355025 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.355008 2583 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 07:14:53.355104 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.355047 2583 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" Nov 04 07:14:53.427608 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.427576 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 07:14:53.429513 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.428711 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 07:14:53.429513 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.428731 2583 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 07:14:53.429513 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.428750 2583 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 07:14:53.429513 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.428780 2583 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Nov 04 07:14:53.432721 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.432703 2583 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 07:14:53.454839 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.454775 2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 07:14:53.455554 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.455540 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 07:14:53.455610 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.455567 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 07:14:53.455610 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.455577 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 07:14:53.455610 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.455599 2583 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.462369 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.462355 2583 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.462449 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.462375 2583 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-6.us-east-2.compute.internal\": node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" Nov 04 07:14:53.477962 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.477946 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" Nov 04 07:14:53.529430 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.529412 2583 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal"] Nov 04 07:14:53.529529 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.529461 2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 07:14:53.530240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.530228 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 07:14:53.530295 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.530256 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 07:14:53.530295 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.530267 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 07:14:53.532392 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.532378 2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 07:14:53.532588 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.532574 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.532657 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.532615 2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 07:14:53.533093 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.533076 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 07:14:53.533216 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.533108 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 07:14:53.533216 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.533119 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 07:14:53.533216 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.533076 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 07:14:53.533216 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.533179 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 07:14:53.533216 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.533192 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 07:14:53.535221 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.535207 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.535304 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.535228 2583 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 07:14:53.535911 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.535891 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 07:14:53.535971 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.535919 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 07:14:53.535971 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.535933 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 07:14:53.556852 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.556838 2583 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" node="ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.558874 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.558853 2583 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" node="ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.578418 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.578402 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" Nov 04 07:14:53.588522 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.588495 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b58482bdf311c221bced295e67dfb26d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b58482bdf311c221bced295e67dfb26d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.588586 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.588530 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b58482bdf311c221bced295e67dfb26d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b58482bdf311c221bced295e67dfb26d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.588586 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.588545 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7ad388a799a062d464456d3dcd4ae6a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b7ad388a799a062d464456d3dcd4ae6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.678790 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.678767 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" Nov 04 07:14:53.689201 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.689180 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b58482bdf311c221bced295e67dfb26d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b58482bdf311c221bced295e67dfb26d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.689286 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.689205 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b58482bdf311c221bced295e67dfb26d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b58482bdf311c221bced295e67dfb26d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.689286 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.689213 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b58482bdf311c221bced295e67dfb26d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b58482bdf311c221bced295e67dfb26d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.689286 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.689240 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b58482bdf311c221bced295e67dfb26d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b58482bdf311c221bced295e67dfb26d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.689286 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.689264 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7ad388a799a062d464456d3dcd4ae6a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b7ad388a799a062d464456d3dcd4ae6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.689436 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.689314 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7ad388a799a062d464456d3dcd4ae6a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b7ad388a799a062d464456d3dcd4ae6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.779003 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.778950 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" Nov 04 07:14:53.860538 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.860518 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.862631 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:53.862614 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:53.879833 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.879810 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" Nov 04 07:14:53.980413 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:53.980384 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" Nov 04 07:14:54.080976 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:54.080917 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" Nov 04 07:14:54.181491 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:54.181468 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" Nov 04 07:14:54.221994 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:54.221972 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Nov 04 07:14:54.222414 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:54.222135 2583 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 04 07:14:54.282613 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:54.282432 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" Nov 04 07:14:54.305431 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:54.305353 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 07:09:54 +0000 UTC" deadline="2028-05-23 19:23:34.237240001 +0000 UTC" Nov 04 07:14:54.305431 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:54.305381 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13596h8m39.931862315s" Nov 04 07:14:54.383642 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:54.383587 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" Nov 04 07:14:54.418468 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:54.418444 2583 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 07:14:54.484743 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:54.484713 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" Nov 04 07:14:54.556424 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:54.556386 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7ad388a799a062d464456d3dcd4ae6a.slice/crio-74dde8637d7b8fd440905c117fd8e135e7a24ffb9b04c353c7920db3f179b5b2 WatchSource:0}: Error finding container 74dde8637d7b8fd440905c117fd8e135e7a24ffb9b04c353c7920db3f179b5b2: Status 404 returned error can't find the container with id 74dde8637d7b8fd440905c117fd8e135e7a24ffb9b04c353c7920db3f179b5b2 Nov 04 07:14:54.556649 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:54.556627 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb58482bdf311c221bced295e67dfb26d.slice/crio-4c6a17b097876c7082ce87b31d331614041295afbbb1e5d1d28f674afc0e94b7 WatchSource:0}: Error finding container 4c6a17b097876c7082ce87b31d331614041295afbbb1e5d1d28f674afc0e94b7: Status 404 returned error can't find the container with id 4c6a17b097876c7082ce87b31d331614041295afbbb1e5d1d28f674afc0e94b7 Nov 04 07:14:54.562862 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:54.562840 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 07:14:54.585575 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:54.585541 2583 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" Nov 04 07:14:54.601434 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:54.601412 2583 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 07:14:54.685469 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:54.685410 2583 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:54.698998 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:54.698980 2583 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 07:14:54.701110 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:54.701095 2583 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:14:54.711099 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:54.711082 2583 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 07:14:54.756586 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:54.756551 2583 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 07:14:55.224561 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.224526 2583 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 07:14:55.261671 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.261641 2583 apiserver.go:52] "Watching apiserver" Nov 04 07:14:55.270469 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.270442 2583 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 07:14:55.271227 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.270864 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-s8gfp","openshift-ovn-kubernetes/ovnkube-node-95dnd","kube-system/global-pull-secret-syncer-m7gt8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal","openshift-network-diagnostics/network-check-target-9qgxt","openshift-image-registry/node-ca-sd2bs","openshift-multus/multus-additional-cni-plugins-grc4v","kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal","openshift-dns/node-resolver-x6v9g","openshift-multus/multus-6jbrj","openshift-multus/network-metrics-daemon-ldwvk","kube-system/konnectivity-agent-2s2xx","openshift-cluster-node-tuning-operator/tuned-dnvwf"] Nov 04 07:14:55.273724 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.273701 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:14:55.276172 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.276146 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.278466 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.278443 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:14:55.278594 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:55.278520 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:14:55.279971 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.279918 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 07:14:55.280083 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.279987 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 07:14:55.281335 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.280661 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.281457 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.281425 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 07:14:55.281729 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.281707 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gcxz6\"" Nov 04 07:14:55.283519 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.282982 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 07:14:55.283519 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.283382 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.285701 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.285677 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:14:55.285802 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:55.285763 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:14:55.286126 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.286110 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Nov 04 07:14:55.289597 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.289561 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.291645 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.291628 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2s2xx" Nov 04 07:14:55.295093 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.294351 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.296851 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.296823 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:14:55.298817 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.298771 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-netns\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.298926 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.298851 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-registration-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.298926 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.298886 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-sys\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.298926 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.298914 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vnql\" (UniqueName: \"kubernetes.io/projected/80a4a0ad-5738-454b-9818-16b544aa5284-kube-api-access-6vnql\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.299137 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299014 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klgh7\" (UniqueName: \"kubernetes.io/projected/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-kube-api-access-klgh7\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.299137 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299048 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fa9b645b-69d5-47a7-8457-409364209023-kubelet-config\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:14:55.299137 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299078 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f33f6c60-eff2-40c2-9350-ba10eaf5442d-host\") pod \"node-ca-sd2bs\" (UID: \"f33f6c60-eff2-40c2-9350-ba10eaf5442d\") " pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:14:55.299137 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299103 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-sys-fs\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.299137 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299130 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2b6n\" (UniqueName: \"kubernetes.io/projected/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-kube-api-access-b2b6n\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.299408 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299153 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysconfig\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.299408 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299203 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-kubernetes\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.299408 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299227 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.299408 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299258 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-cnibin\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.299408 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299287 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-cni-binary-copy\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.299408 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299303 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-cni-multus\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.299408 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299338 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-daemon-config\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.299408 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299364 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fa9b645b-69d5-47a7-8457-409364209023-dbus\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:14:55.299408 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299389 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqkrc\" (UniqueName: \"kubernetes.io/projected/f33f6c60-eff2-40c2-9350-ba10eaf5442d-kube-api-access-bqkrc\") pod \"node-ca-sd2bs\" (UID: \"f33f6c60-eff2-40c2-9350-ba10eaf5442d\") " pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:14:55.300066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299431 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.300066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299482 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-system-cni-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.300066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299529 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-socket-dir-parent\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.300066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299560 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-k8s-cni-cncf-io\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.300066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299581 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-kubelet\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.300066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299631 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhnsv\" (UniqueName: \"kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv\") pod \"network-check-target-9qgxt\" (UID: \"97707149-0248-45a0-87d2-7347b960c2db\") " pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:14:55.300066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299639 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:14:55.300066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299662 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-device-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.300066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299692 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-modprobe-d\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.300066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299719 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-cni-bin\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.300066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299771 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-hostroot\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.300066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299802 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-conf-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.300066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299851 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-etc-kubernetes\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.300066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299880 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:14:55.300066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299922 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysctl-conf\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.300066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.299948 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-tuned\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.300066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.300007 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t49f8\" (UniqueName: \"kubernetes.io/projected/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-kube-api-access-t49f8\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.301223 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.300031 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-socket-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.301223 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.300217 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysctl-d\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.301223 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.300256 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-systemd\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.301223 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.300287 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-run\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.301223 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.300325 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-lib-modules\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.301223 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.300369 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-host\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.301223 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.300392 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-tmp\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.301223 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.300438 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-system-cni-dir\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.301223 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.300468 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-multus-certs\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.301223 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.300529 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-cnibin\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.301223 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.300561 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-cni-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.301223 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.300666 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f33f6c60-eff2-40c2-9350-ba10eaf5442d-serviceca\") pod \"node-ca-sd2bs\" (UID: \"f33f6c60-eff2-40c2-9350-ba10eaf5442d\") " pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:14:55.301223 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.300715 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-var-lib-kubelet\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.301223 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.300771 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-cni-binary-copy\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.301223 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.300826 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-tuning-conf-dir\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.301223 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.300872 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.302333 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.301014 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.305091 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.305057 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:14:55.305193 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.305136 2583 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 07:14:55.305193 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:55.305145 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:14:55.332382 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.332362 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9zpj5" Nov 04 07:14:55.344906 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.344873 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9zpj5" Nov 04 07:14:55.387217 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.387193 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 07:14:55.387445 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.387411 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 07:14:55.387995 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.387915 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 07:14:55.387995 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.387918 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 07:14:55.388151 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.388134 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 07:14:55.388358 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.388339 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 07:14:55.388463 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.388389 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 07:14:55.388463 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.388430 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 07:14:55.388956 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.388917 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 07:14:55.389043 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.389001 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 07:14:55.389107 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.389065 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 07:14:55.389107 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.389100 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 07:14:55.389336 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.389321 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 07:14:55.389419 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.389402 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 07:14:55.389528 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.389513 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 07:14:55.389631 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.389611 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-slv86\"" Nov 04 07:14:55.390349 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.390302 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8bggl\"" Nov 04 07:14:55.390349 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.390305 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 07:14:55.390488 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.390366 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 07:14:55.390488 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.390452 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 07:14:55.390584 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.390537 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 07:14:55.390644 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.390595 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7qf26\"" Nov 04 07:14:55.391443 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.390909 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tqngx\"" Nov 04 07:14:55.391443 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.390977 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-8kctk\"" Nov 04 07:14:55.391443 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.391096 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ltv7r\"" Nov 04 07:14:55.391443 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.391141 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 07:14:55.391443 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.391195 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 07:14:55.391443 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.391248 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-tjts4\"" Nov 04 07:14:55.391443 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.391282 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 07:14:55.391443 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.391354 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 07:14:55.391443 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.391403 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l4l9x\"" Nov 04 07:14:55.392932 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.392913 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 07:14:55.402004 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.401622 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysctl-conf\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.402004 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.401657 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.402004 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.401692 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.402004 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.401721 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovnkube-script-lib\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.402004 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.401745 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-cni-binary-copy\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.402004 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.401768 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-cni-multus\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.402004 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.401792 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-daemon-config\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.402004 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.401816 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.402004 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.401834 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysctl-conf\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.402004 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.401840 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-systemd\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.402004 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.401890 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9-konnectivity-ca\") pod \"konnectivity-agent-2s2xx\" (UID: \"ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9\") " pod="kube-system/konnectivity-agent-2s2xx" Nov 04 07:14:55.402004 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.401893 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-systemd\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.402004 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.401911 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs\") pod \"network-metrics-daemon-ldwvk\" (UID: \"17916760-be6c-47bf-8569-94ec0e36e304\") " pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:14:55.402004 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.401945 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-cni-multus\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.402004 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.401951 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsljv\" (UniqueName: \"kubernetes.io/projected/17916760-be6c-47bf-8569-94ec0e36e304-kube-api-access-lsljv\") pod \"network-metrics-daemon-ldwvk\" (UID: \"17916760-be6c-47bf-8569-94ec0e36e304\") " pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:14:55.402004 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.401974 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-system-cni-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.402004 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402002 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhnsv\" (UniqueName: \"kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv\") pod \"network-check-target-9qgxt\" (UID: \"97707149-0248-45a0-87d2-7347b960c2db\") " pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:14:55.402978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402029 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-modprobe-d\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.402978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402053 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-tuned\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.402978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402119 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t49f8\" (UniqueName: \"kubernetes.io/projected/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-kube-api-access-t49f8\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.402978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402148 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-cnibin\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.402978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402175 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-hostroot\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.402978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402200 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:14:55.402978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402228 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.402978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402254 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysctl-d\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.402978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402288 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-host\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.402978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402308 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-multus-certs\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.402978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402332 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-systemd\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.402978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402356 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-var-lib-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.402978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402364 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.402978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402378 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-env-overrides\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.402978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402400 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f33f6c60-eff2-40c2-9350-ba10eaf5442d-serviceca\") pod \"node-ca-sd2bs\" (UID: \"f33f6c60-eff2-40c2-9350-ba10eaf5442d\") " pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:14:55.402978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402437 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-cni-binary-copy\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.402978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402442 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-cni-binary-copy\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.403608 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402447 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-cnibin\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.403608 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402460 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-tuning-conf-dir\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.403608 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402512 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-hostroot\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.403608 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402662 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysctl-d\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.403608 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402735 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-system-cni-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.403608 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402778 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-host\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.403608 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402813 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-multus-certs\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.403608 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.402816 2583 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 07:14:55.403608 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.403089 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.403608 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.403156 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.403608 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.403204 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-etc-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.403608 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:55.403352 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 07:14:55.403608 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.403448 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-modprobe-d\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.403608 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.403563 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-ovn\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.403608 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.403601 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f33f6c60-eff2-40c2-9350-ba10eaf5442d-serviceca\") pod \"node-ca-sd2bs\" (UID: \"f33f6c60-eff2-40c2-9350-ba10eaf5442d\") " pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:14:55.403608 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.403615 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovn-node-metrics-cert\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.404407 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.403644 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2dxb\" (UniqueName: \"kubernetes.io/projected/19b2ab56-a868-4570-b098-37c517da236a-kube-api-access-f2dxb\") pod \"node-resolver-x6v9g\" (UID: \"19b2ab56-a868-4570-b098-37c517da236a\") " pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:14:55.404407 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.403704 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-registration-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.404407 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.403743 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-sys\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.404407 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:55.403829 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret podName:fa9b645b-69d5-47a7-8457-409364209023 nodeName:}" failed. No retries permitted until 2026-11-04 07:14:55.903782602 +0000 UTC m=+2.993309546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret") pod "global-pull-secret-syncer-m7gt8" (UID: "fa9b645b-69d5-47a7-8457-409364209023") : object "kube-system"/"original-pull-secret" not registered Nov 04 07:14:55.404407 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404062 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63631d09-f1c5-43a6-98e9-4002ada7949a-host-slash\") pod \"iptables-alerter-s8gfp\" (UID: \"63631d09-f1c5-43a6-98e9-4002ada7949a\") " pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:14:55.404407 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404095 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgllt\" (UniqueName: \"kubernetes.io/projected/63631d09-f1c5-43a6-98e9-4002ada7949a-kube-api-access-mgllt\") pod \"iptables-alerter-s8gfp\" (UID: \"63631d09-f1c5-43a6-98e9-4002ada7949a\") " pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:14:55.404407 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404124 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-run-ovn-kubernetes\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.404407 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404156 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.404407 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404166 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-cni-netd\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.404407 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404179 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-cni-binary-copy\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.404407 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404187 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19b2ab56-a868-4570-b098-37c517da236a-tmp-dir\") pod \"node-resolver-x6v9g\" (UID: \"19b2ab56-a868-4570-b098-37c517da236a\") " pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:14:55.404407 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404235 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-sys\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.404407 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404235 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fa9b645b-69d5-47a7-8457-409364209023-kubelet-config\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:14:55.404407 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404272 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f33f6c60-eff2-40c2-9350-ba10eaf5442d-host\") pod \"node-ca-sd2bs\" (UID: \"f33f6c60-eff2-40c2-9350-ba10eaf5442d\") " pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:14:55.404407 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404274 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-tuning-conf-dir\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.404407 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404291 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-sys-fs\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.405240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404297 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fa9b645b-69d5-47a7-8457-409364209023-kubelet-config\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:14:55.405240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404349 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-sys-fs\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.405240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404391 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f33f6c60-eff2-40c2-9350-ba10eaf5442d-host\") pod \"node-ca-sd2bs\" (UID: \"f33f6c60-eff2-40c2-9350-ba10eaf5442d\") " pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:14:55.405240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404434 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-kubernetes\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.405240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404468 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-kubelet\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.405240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404474 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-registration-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.405240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404539 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-slash\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.405240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404545 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-kubernetes\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.405240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404574 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vnql\" (UniqueName: \"kubernetes.io/projected/80a4a0ad-5738-454b-9818-16b544aa5284-kube-api-access-6vnql\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.405240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404607 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.405240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404740 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-cnibin\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.405240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404769 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fa9b645b-69d5-47a7-8457-409364209023-dbus\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:14:55.405240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404831 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqkrc\" (UniqueName: \"kubernetes.io/projected/f33f6c60-eff2-40c2-9350-ba10eaf5442d-kube-api-access-bqkrc\") pod \"node-ca-sd2bs\" (UID: \"f33f6c60-eff2-40c2-9350-ba10eaf5442d\") " pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:14:55.405240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404862 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-systemd-units\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.405240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404895 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-socket-dir-parent\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.405240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404959 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-k8s-cni-cncf-io\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.405240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.404991 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-kubelet\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.406094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405018 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-device-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.406094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405048 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-run-netns\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.406094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405075 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-log-socket\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.406094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405103 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-cni-bin\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.406094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405130 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-conf-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.406094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405205 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-etc-kubernetes\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.406094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405223 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-conf-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.406094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405234 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-system-cni-dir\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.406094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405262 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-cni-bin\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.406094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405292 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-socket-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.406094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405319 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-run\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.406094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405325 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-socket-dir-parent\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.406094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405348 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-lib-modules\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.406094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405374 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-tmp\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.406094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405403 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9-agent-certs\") pod \"konnectivity-agent-2s2xx\" (UID: \"ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9\") " pod="kube-system/konnectivity-agent-2s2xx" Nov 04 07:14:55.406094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405413 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.406094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405432 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtzpm\" (UniqueName: \"kubernetes.io/projected/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-kube-api-access-rtzpm\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405374 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fa9b645b-69d5-47a7-8457-409364209023-dbus\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405526 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/63631d09-f1c5-43a6-98e9-4002ada7949a-iptables-alerter-script\") pod \"iptables-alerter-s8gfp\" (UID: \"63631d09-f1c5-43a6-98e9-4002ada7949a\") " pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405531 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-cni-bin\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405564 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-node-log\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405613 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19b2ab56-a868-4570-b098-37c517da236a-hosts-file\") pod \"node-resolver-x6v9g\" (UID: \"19b2ab56-a868-4570-b098-37c517da236a\") " pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405644 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-cni-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405655 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-lib-modules\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405675 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-var-lib-kubelet\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405699 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-system-cni-dir\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405716 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-netns\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405745 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovnkube-config\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405801 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-cnibin\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.405804 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-socket-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.406009 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-etc-kubernetes\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.406048 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-k8s-cni-cncf-io\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.406084 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-kubelet\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.406117 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-var-lib-kubelet\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.408071 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.406122 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-netns\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.408984 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.406156 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klgh7\" (UniqueName: \"kubernetes.io/projected/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-kube-api-access-klgh7\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.408984 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.406175 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-device-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.408984 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.406190 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2b6n\" (UniqueName: \"kubernetes.io/projected/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-kube-api-access-b2b6n\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.408984 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.406240 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-run\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.408984 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.406277 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-cni-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.408984 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.406313 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysconfig\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.408984 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.406399 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysconfig\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.408984 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.407399 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-daemon-config\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.408984 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.408434 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-tmp\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.414603 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.414580 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-tuned\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.430360 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.430331 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqkrc\" (UniqueName: \"kubernetes.io/projected/f33f6c60-eff2-40c2-9350-ba10eaf5442d-kube-api-access-bqkrc\") pod \"node-ca-sd2bs\" (UID: \"f33f6c60-eff2-40c2-9350-ba10eaf5442d\") " pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:14:55.430360 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.430354 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2b6n\" (UniqueName: \"kubernetes.io/projected/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-kube-api-access-b2b6n\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.434513 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:55.434486 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 07:14:55.434627 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:55.434516 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 07:14:55.434627 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:55.434546 2583 projected.go:196] Error preparing data for projected volume kube-api-access-nhnsv for pod openshift-network-diagnostics/network-check-target-9qgxt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 07:14:55.434756 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:55.434642 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv podName:97707149-0248-45a0-87d2-7347b960c2db nodeName:}" failed. No retries permitted until 2026-11-04 07:14:55.934615356 +0000 UTC m=+3.024142296 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nhnsv" (UniqueName: "kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv") pod "network-check-target-9qgxt" (UID: "97707149-0248-45a0-87d2-7347b960c2db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 07:14:55.437567 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.437538 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vnql\" (UniqueName: \"kubernetes.io/projected/80a4a0ad-5738-454b-9818-16b544aa5284-kube-api-access-6vnql\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.437767 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.437613 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t49f8\" (UniqueName: \"kubernetes.io/projected/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-kube-api-access-t49f8\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.438372 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.438320 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" event={"ID":"b58482bdf311c221bced295e67dfb26d","Type":"ContainerStarted","Data":"4c6a17b097876c7082ce87b31d331614041295afbbb1e5d1d28f674afc0e94b7"} Nov 04 07:14:55.439023 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.439001 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klgh7\" (UniqueName: \"kubernetes.io/projected/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-kube-api-access-klgh7\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.439127 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.439080 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" event={"ID":"b7ad388a799a062d464456d3dcd4ae6a","Type":"ContainerStarted","Data":"74dde8637d7b8fd440905c117fd8e135e7a24ffb9b04c353c7920db3f179b5b2"} Nov 04 07:14:55.507353 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507290 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-cni-bin\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.507353 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507327 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9-agent-certs\") pod \"konnectivity-agent-2s2xx\" (UID: \"ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9\") " pod="kube-system/konnectivity-agent-2s2xx" Nov 04 07:14:55.507541 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507352 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtzpm\" (UniqueName: \"kubernetes.io/projected/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-kube-api-access-rtzpm\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.507541 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507390 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/63631d09-f1c5-43a6-98e9-4002ada7949a-iptables-alerter-script\") pod \"iptables-alerter-s8gfp\" (UID: \"63631d09-f1c5-43a6-98e9-4002ada7949a\") " pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:14:55.507541 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507421 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-node-log\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.507541 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507445 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19b2ab56-a868-4570-b098-37c517da236a-hosts-file\") pod \"node-resolver-x6v9g\" (UID: \"19b2ab56-a868-4570-b098-37c517da236a\") " pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:14:55.507541 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507470 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovnkube-config\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.507541 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507500 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.507541 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507542 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovnkube-script-lib\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.507854 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507571 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9-konnectivity-ca\") pod \"konnectivity-agent-2s2xx\" (UID: \"ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9\") " pod="kube-system/konnectivity-agent-2s2xx" Nov 04 07:14:55.507854 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507596 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs\") pod \"network-metrics-daemon-ldwvk\" (UID: \"17916760-be6c-47bf-8569-94ec0e36e304\") " pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:14:55.507854 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507623 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsljv\" (UniqueName: \"kubernetes.io/projected/17916760-be6c-47bf-8569-94ec0e36e304-kube-api-access-lsljv\") pod \"network-metrics-daemon-ldwvk\" (UID: \"17916760-be6c-47bf-8569-94ec0e36e304\") " pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:14:55.507854 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507699 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.507854 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507726 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-systemd\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.507854 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507750 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-var-lib-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.507854 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507774 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-env-overrides\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.507854 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507805 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-etc-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.507854 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507838 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-ovn\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.508285 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507863 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovn-node-metrics-cert\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.508285 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507889 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2dxb\" (UniqueName: \"kubernetes.io/projected/19b2ab56-a868-4570-b098-37c517da236a-kube-api-access-f2dxb\") pod \"node-resolver-x6v9g\" (UID: \"19b2ab56-a868-4570-b098-37c517da236a\") " pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:14:55.508285 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507916 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63631d09-f1c5-43a6-98e9-4002ada7949a-host-slash\") pod \"iptables-alerter-s8gfp\" (UID: \"63631d09-f1c5-43a6-98e9-4002ada7949a\") " pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:14:55.508285 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507941 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgllt\" (UniqueName: \"kubernetes.io/projected/63631d09-f1c5-43a6-98e9-4002ada7949a-kube-api-access-mgllt\") pod \"iptables-alerter-s8gfp\" (UID: \"63631d09-f1c5-43a6-98e9-4002ada7949a\") " pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:14:55.508285 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507962 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-cni-bin\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.508285 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.508023 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-run-ovn-kubernetes\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.508285 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.508043 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19b2ab56-a868-4570-b098-37c517da236a-hosts-file\") pod \"node-resolver-x6v9g\" (UID: \"19b2ab56-a868-4570-b098-37c517da236a\") " pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:14:55.508285 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.508058 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-node-log\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.508285 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.508103 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.508285 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.507966 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-run-ovn-kubernetes\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.508285 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.508146 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-cni-netd\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.508285 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.508164 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19b2ab56-a868-4570-b098-37c517da236a-tmp-dir\") pod \"node-resolver-x6v9g\" (UID: \"19b2ab56-a868-4570-b098-37c517da236a\") " pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:14:55.508285 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.508183 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-kubelet\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.508285 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.508197 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-slash\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.508285 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.508217 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-systemd-units\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.508285 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.508236 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-run-netns\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.508285 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.508251 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-log-socket\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.509123 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.508311 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-log-socket\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.509123 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.508601 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovnkube-script-lib\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.509123 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.508722 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-env-overrides\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.509123 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.508918 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/63631d09-f1c5-43a6-98e9-4002ada7949a-iptables-alerter-script\") pod \"iptables-alerter-s8gfp\" (UID: \"63631d09-f1c5-43a6-98e9-4002ada7949a\") " pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:14:55.509123 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.508987 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-cni-netd\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.509123 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.509005 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovnkube-config\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.509123 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.509055 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-etc-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.509123 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.509088 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-ovn\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.509381 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.509270 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9-konnectivity-ca\") pod \"konnectivity-agent-2s2xx\" (UID: \"ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9\") " pod="kube-system/konnectivity-agent-2s2xx" Nov 04 07:14:55.509381 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.509301 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19b2ab56-a868-4570-b098-37c517da236a-tmp-dir\") pod \"node-resolver-x6v9g\" (UID: \"19b2ab56-a868-4570-b098-37c517da236a\") " pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:14:55.509381 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.509349 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-kubelet\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.509381 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:55.509363 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 07:14:55.509561 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.509387 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-slash\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.509561 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:55.509427 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs podName:17916760-be6c-47bf-8569-94ec0e36e304 nodeName:}" failed. No retries permitted until 2026-11-04 07:14:56.00940949 +0000 UTC m=+3.098936427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs") pod "network-metrics-daemon-ldwvk" (UID: "17916760-be6c-47bf-8569-94ec0e36e304") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 07:14:55.509561 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.509432 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-run-netns\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.509561 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.509440 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63631d09-f1c5-43a6-98e9-4002ada7949a-host-slash\") pod \"iptables-alerter-s8gfp\" (UID: \"63631d09-f1c5-43a6-98e9-4002ada7949a\") " pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:14:55.509561 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.509489 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-systemd-units\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.509561 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.509533 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-systemd\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.509836 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.509618 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-var-lib-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.509836 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.509620 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.510598 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.510577 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9-agent-certs\") pod \"konnectivity-agent-2s2xx\" (UID: \"ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9\") " pod="kube-system/konnectivity-agent-2s2xx" Nov 04 07:14:55.511363 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.511341 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovn-node-metrics-cert\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.527377 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.527355 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtzpm\" (UniqueName: \"kubernetes.io/projected/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-kube-api-access-rtzpm\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.528064 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.528044 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgllt\" (UniqueName: \"kubernetes.io/projected/63631d09-f1c5-43a6-98e9-4002ada7949a-kube-api-access-mgllt\") pod \"iptables-alerter-s8gfp\" (UID: \"63631d09-f1c5-43a6-98e9-4002ada7949a\") " pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:14:55.528697 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.528658 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsljv\" (UniqueName: \"kubernetes.io/projected/17916760-be6c-47bf-8569-94ec0e36e304-kube-api-access-lsljv\") pod \"network-metrics-daemon-ldwvk\" (UID: \"17916760-be6c-47bf-8569-94ec0e36e304\") " pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:14:55.531306 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.531285 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2dxb\" (UniqueName: \"kubernetes.io/projected/19b2ab56-a868-4570-b098-37c517da236a-kube-api-access-f2dxb\") pod \"node-resolver-x6v9g\" (UID: \"19b2ab56-a868-4570-b098-37c517da236a\") " pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:14:55.590677 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.590651 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:14:55.600921 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.600902 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6jbrj" Nov 04 07:14:55.602115 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:55.602084 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf33f6c60_eff2_40c2_9350_ba10eaf5442d.slice/crio-5fc708252796083f69ed7f8766d4018af524ce264be29bba95ae7dac9aa83d4e WatchSource:0}: Error finding container 5fc708252796083f69ed7f8766d4018af524ce264be29bba95ae7dac9aa83d4e: Status 404 returned error can't find the container with id 5fc708252796083f69ed7f8766d4018af524ce264be29bba95ae7dac9aa83d4e Nov 04 07:14:55.610970 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:55.610949 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d4ba0b2_5841_4bdc_88b1_4ee4b9c7db8f.slice/crio-709bfc975391d29847dce94faba2126dbd67b2d87a85fc353b0f0245dd41f34e WatchSource:0}: Error finding container 709bfc975391d29847dce94faba2126dbd67b2d87a85fc353b0f0245dd41f34e: Status 404 returned error can't find the container with id 709bfc975391d29847dce94faba2126dbd67b2d87a85fc353b0f0245dd41f34e Nov 04 07:14:55.616229 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.616211 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:14:55.620822 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.620803 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:14:55.625607 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:55.625571 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa408a9f_2280_4a77_bddd_c31dbe7fdb9c.slice/crio-5b287a145f727b1a4a1a27b9710ee3ff2ed4cf8a7797b20a959e36933e317fc8 WatchSource:0}: Error finding container 5b287a145f727b1a4a1a27b9710ee3ff2ed4cf8a7797b20a959e36933e317fc8: Status 404 returned error can't find the container with id 5b287a145f727b1a4a1a27b9710ee3ff2ed4cf8a7797b20a959e36933e317fc8 Nov 04 07:14:55.632633 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:55.632603 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a43577d_3b09_4f31_89ec_5b4b7f7efb4c.slice/crio-586241635cfe2eb35d814b48f13ed6bc1eb61911c2420302149ec75e6bd04383 WatchSource:0}: Error finding container 586241635cfe2eb35d814b48f13ed6bc1eb61911c2420302149ec75e6bd04383: Status 404 returned error can't find the container with id 586241635cfe2eb35d814b48f13ed6bc1eb61911c2420302149ec75e6bd04383 Nov 04 07:14:55.644727 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.644701 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2s2xx" Nov 04 07:14:55.646816 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.646795 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:14:55.658434 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.658412 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:14:55.659509 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:55.659475 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a4a0ad_5738_454b_9818_16b544aa5284.slice/crio-42519616ec898f3cc24635c600c6319af9749812f96791d88ba1f4dbd5823264 WatchSource:0}: Error finding container 42519616ec898f3cc24635c600c6319af9749812f96791d88ba1f4dbd5823264: Status 404 returned error can't find the container with id 42519616ec898f3cc24635c600c6319af9749812f96791d88ba1f4dbd5823264 Nov 04 07:14:55.660939 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:55.660909 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae126ffc_5714_4d59_b4ad_6f2baa0cf1a9.slice/crio-e831a99c631e426e454e7baf13c0458c225215cf935034bd771343ed086fc2c4 WatchSource:0}: Error finding container e831a99c631e426e454e7baf13c0458c225215cf935034bd771343ed086fc2c4: Status 404 returned error can't find the container with id e831a99c631e426e454e7baf13c0458c225215cf935034bd771343ed086fc2c4 Nov 04 07:14:55.668218 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.668046 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:14:55.670607 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.670542 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:14:55.683580 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:14:55.683445 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30ff434f_8151_47d7_b6e9_51d9a3d8d6ad.slice/crio-192706c9c89870c90332435973b8d137b0548ea06c810e67817775533b83ccc9 WatchSource:0}: Error finding container 192706c9c89870c90332435973b8d137b0548ea06c810e67817775533b83ccc9: Status 404 returned error can't find the container with id 192706c9c89870c90332435973b8d137b0548ea06c810e67817775533b83ccc9 Nov 04 07:14:55.911570 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:55.911471 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:14:55.911736 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:55.911655 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 07:14:55.911736 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:55.911722 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret podName:fa9b645b-69d5-47a7-8457-409364209023 nodeName:}" failed. No retries permitted until 2026-11-04 07:14:56.911702532 +0000 UTC m=+4.001229458 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret") pod "global-pull-secret-syncer-m7gt8" (UID: "fa9b645b-69d5-47a7-8457-409364209023") : object "kube-system"/"original-pull-secret" not registered Nov 04 07:14:56.012502 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:56.012471 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs\") pod \"network-metrics-daemon-ldwvk\" (UID: \"17916760-be6c-47bf-8569-94ec0e36e304\") " pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:14:56.012696 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:56.012516 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhnsv\" (UniqueName: \"kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv\") pod \"network-check-target-9qgxt\" (UID: \"97707149-0248-45a0-87d2-7347b960c2db\") " pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:14:56.012696 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:56.012664 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 07:14:56.012696 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:56.012687 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 07:14:56.012860 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:56.012706 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 07:14:56.012860 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:56.012719 2583 projected.go:196] Error preparing data for projected volume kube-api-access-nhnsv for pod openshift-network-diagnostics/network-check-target-9qgxt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 07:14:56.012860 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:56.012739 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs podName:17916760-be6c-47bf-8569-94ec0e36e304 nodeName:}" failed. No retries permitted until 2026-11-04 07:14:57.012718286 +0000 UTC m=+4.102245205 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs") pod "network-metrics-daemon-ldwvk" (UID: "17916760-be6c-47bf-8569-94ec0e36e304") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 07:14:56.012860 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:56.012764 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv podName:97707149-0248-45a0-87d2-7347b960c2db nodeName:}" failed. No retries permitted until 2026-11-04 07:14:57.012749891 +0000 UTC m=+4.102276809 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nhnsv" (UniqueName: "kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv") pod "network-check-target-9qgxt" (UID: "97707149-0248-45a0-87d2-7347b960c2db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 07:14:56.347760 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:56.347644 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 07:09:55 +0000 UTC" deadline="2028-07-11 22:55:49.70857356 +0000 UTC" Nov 04 07:14:56.347760 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:56.347707 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14775h40m53.360870999s" Nov 04 07:14:56.429785 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:56.429743 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:14:56.429967 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:56.429899 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:14:56.452861 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:56.452692 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" event={"ID":"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c","Type":"ContainerStarted","Data":"5b287a145f727b1a4a1a27b9710ee3ff2ed4cf8a7797b20a959e36933e317fc8"} Nov 04 07:14:56.459763 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:56.459638 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" event={"ID":"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad","Type":"ContainerStarted","Data":"192706c9c89870c90332435973b8d137b0548ea06c810e67817775533b83ccc9"} Nov 04 07:14:56.461982 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:56.461907 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2s2xx" event={"ID":"ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9","Type":"ContainerStarted","Data":"e831a99c631e426e454e7baf13c0458c225215cf935034bd771343ed086fc2c4"} Nov 04 07:14:56.465166 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:56.465120 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grc4v" event={"ID":"80a4a0ad-5738-454b-9818-16b544aa5284","Type":"ContainerStarted","Data":"42519616ec898f3cc24635c600c6319af9749812f96791d88ba1f4dbd5823264"} Nov 04 07:14:56.469588 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:56.469404 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6jbrj" event={"ID":"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f","Type":"ContainerStarted","Data":"709bfc975391d29847dce94faba2126dbd67b2d87a85fc353b0f0245dd41f34e"} Nov 04 07:14:56.486365 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:56.486334 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sd2bs" event={"ID":"f33f6c60-eff2-40c2-9350-ba10eaf5442d","Type":"ContainerStarted","Data":"5fc708252796083f69ed7f8766d4018af524ce264be29bba95ae7dac9aa83d4e"} Nov 04 07:14:56.488850 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:56.488819 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s8gfp" event={"ID":"63631d09-f1c5-43a6-98e9-4002ada7949a","Type":"ContainerStarted","Data":"a2a9454ff0a9aaee20e782323e0e1f9f2db0f10bed694f3003567300d327e1ec"} Nov 04 07:14:56.492354 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:56.492272 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x6v9g" event={"ID":"19b2ab56-a868-4570-b098-37c517da236a","Type":"ContainerStarted","Data":"0920f6d09f5542e9ff46a73d1e32487fa63e11fe8640270165d7c37419792f2b"} Nov 04 07:14:56.495438 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:56.495400 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" event={"ID":"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c","Type":"ContainerStarted","Data":"586241635cfe2eb35d814b48f13ed6bc1eb61911c2420302149ec75e6bd04383"} Nov 04 07:14:56.920306 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:56.920241 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:14:56.920524 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:56.920394 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 07:14:56.920524 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:56.920455 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret podName:fa9b645b-69d5-47a7-8457-409364209023 nodeName:}" failed. No retries permitted until 2026-11-04 07:14:58.920437215 +0000 UTC m=+6.009964135 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret") pod "global-pull-secret-syncer-m7gt8" (UID: "fa9b645b-69d5-47a7-8457-409364209023") : object "kube-system"/"original-pull-secret" not registered Nov 04 07:14:57.020592 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:57.020555 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs\") pod \"network-metrics-daemon-ldwvk\" (UID: \"17916760-be6c-47bf-8569-94ec0e36e304\") " pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:14:57.020592 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:57.020608 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhnsv\" (UniqueName: \"kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv\") pod \"network-check-target-9qgxt\" (UID: \"97707149-0248-45a0-87d2-7347b960c2db\") " pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:14:57.020847 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:57.020814 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 07:14:57.020847 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:57.020832 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 07:14:57.020847 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:57.020846 2583 projected.go:196] Error preparing data for projected volume kube-api-access-nhnsv for pod openshift-network-diagnostics/network-check-target-9qgxt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 07:14:57.021012 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:57.020902 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv podName:97707149-0248-45a0-87d2-7347b960c2db nodeName:}" failed. No retries permitted until 2026-11-04 07:14:59.020884025 +0000 UTC m=+6.110410949 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nhnsv" (UniqueName: "kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv") pod "network-check-target-9qgxt" (UID: "97707149-0248-45a0-87d2-7347b960c2db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 07:14:57.021334 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:57.021312 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 07:14:57.021422 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:57.021376 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs podName:17916760-be6c-47bf-8569-94ec0e36e304 nodeName:}" failed. No retries permitted until 2026-11-04 07:14:59.021357701 +0000 UTC m=+6.110884645 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs") pod "network-metrics-daemon-ldwvk" (UID: "17916760-be6c-47bf-8569-94ec0e36e304") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 07:14:57.348668 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:57.348569 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 07:09:55 +0000 UTC" deadline="2028-06-28 19:57:10.092610508 +0000 UTC" Nov 04 07:14:57.348668 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:57.348615 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14460h42m12.743999803s" Nov 04 07:14:57.432622 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:57.432573 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:14:57.432821 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:57.432736 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:14:57.433152 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:57.433129 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:14:57.433246 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:57.433226 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:14:57.506769 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:57.506707 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" event={"ID":"b7ad388a799a062d464456d3dcd4ae6a","Type":"ContainerStarted","Data":"3c255199ff55b8522bbb1ec55cadec2ea05872ba792c597e9e11b2f9fa9605c1"} Nov 04 07:14:57.525413 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:57.525341 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" podStartSLOduration=2.525321151 podStartE2EDuration="2.525321151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:14:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:14:57.524298993 +0000 UTC m=+4.613825938" watchObservedRunningTime="2026-11-04 07:14:57.525321151 +0000 UTC m=+4.614848097" Nov 04 07:14:58.429779 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:58.429736 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:14:58.430240 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:58.429873 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:14:58.937539 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:58.937163 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:14:58.937682 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:58.937443 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 07:14:58.937773 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:58.937750 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret podName:fa9b645b-69d5-47a7-8457-409364209023 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:02.93772402 +0000 UTC m=+10.027250941 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret") pod "global-pull-secret-syncer-m7gt8" (UID: "fa9b645b-69d5-47a7-8457-409364209023") : object "kube-system"/"original-pull-secret" not registered Nov 04 07:14:59.038070 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:59.038034 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs\") pod \"network-metrics-daemon-ldwvk\" (UID: \"17916760-be6c-47bf-8569-94ec0e36e304\") " pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:14:59.038187 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:59.038091 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhnsv\" (UniqueName: \"kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv\") pod \"network-check-target-9qgxt\" (UID: \"97707149-0248-45a0-87d2-7347b960c2db\") " pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:14:59.038300 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:59.038283 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 07:14:59.038354 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:59.038313 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 07:14:59.038354 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:59.038330 2583 projected.go:196] Error preparing data for projected volume kube-api-access-nhnsv for pod openshift-network-diagnostics/network-check-target-9qgxt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 07:14:59.038461 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:59.038384 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv podName:97707149-0248-45a0-87d2-7347b960c2db nodeName:}" failed. No retries permitted until 2026-11-04 07:15:03.038366039 +0000 UTC m=+10.127892980 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nhnsv" (UniqueName: "kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv") pod "network-check-target-9qgxt" (UID: "97707149-0248-45a0-87d2-7347b960c2db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 07:14:59.038728 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:59.038701 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 07:14:59.038802 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:59.038757 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs podName:17916760-be6c-47bf-8569-94ec0e36e304 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:03.038740003 +0000 UTC m=+10.128266949 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs") pod "network-metrics-daemon-ldwvk" (UID: "17916760-be6c-47bf-8569-94ec0e36e304") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 07:14:59.438420 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:59.434864 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:14:59.438420 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:59.434879 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:14:59.438420 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:59.435239 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:14:59.438420 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:14:59.435044 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:14:59.520630 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:59.515812 2583 generic.go:356] "Generic (PLEG): container finished" podID="b58482bdf311c221bced295e67dfb26d" containerID="f5208030fa712169bbac43c893eb74643314eda131eb62333e9ef8bc83bf153b" exitCode=0 Nov 04 07:14:59.520630 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:59.515910 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" event={"ID":"b58482bdf311c221bced295e67dfb26d","Type":"ContainerDied","Data":"f5208030fa712169bbac43c893eb74643314eda131eb62333e9ef8bc83bf153b"} Nov 04 07:14:59.521570 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:59.521254 2583 generic.go:356] "Generic (PLEG): container finished" podID="80a4a0ad-5738-454b-9818-16b544aa5284" containerID="9588d795a7ddd92e5ad48127061801edc96b481a3bde7f723ce0cf9b538d804a" exitCode=0 Nov 04 07:14:59.521570 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:14:59.521295 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grc4v" event={"ID":"80a4a0ad-5738-454b-9818-16b544aa5284","Type":"ContainerDied","Data":"9588d795a7ddd92e5ad48127061801edc96b481a3bde7f723ce0cf9b538d804a"} Nov 04 07:15:00.429982 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:00.429390 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:00.429982 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:00.429548 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:00.530787 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:00.530745 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" event={"ID":"b58482bdf311c221bced295e67dfb26d","Type":"ContainerStarted","Data":"73b30ee9b7da7a34be4a34e34e562bde1a77ac6d2e6a095991ffae320b9c8c11"} Nov 04 07:15:00.540761 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:00.540699 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2s2xx" event={"ID":"ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9","Type":"ContainerStarted","Data":"a71dc0af6fddc77add1af2dcec25bb4d96c890d988c06fce47a76ede90277db3"} Nov 04 07:15:00.551529 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:00.551478 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" podStartSLOduration=5.551463904 podStartE2EDuration="5.551463904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:14:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:00.550441854 +0000 UTC m=+7.639968795" watchObservedRunningTime="2026-11-04 07:15:00.551463904 +0000 UTC m=+7.640990848" Nov 04 07:15:01.431708 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:01.431673 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:01.431900 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:01.431812 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:01.432227 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:01.432204 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:01.432326 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:01.432311 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:01.547939 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:01.547899 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-sd2bs" event={"ID":"f33f6c60-eff2-40c2-9350-ba10eaf5442d","Type":"ContainerStarted","Data":"35ba0bef9b7f1ef2a1a85ea2b4bd6854108224d23d1d8ddb002b2014d31f7834"} Nov 04 07:15:01.565485 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:01.565415 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2s2xx" podStartSLOduration=3.644698903 podStartE2EDuration="7.565394262s" totalImagesPullingTime="3.920695359s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:14:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:00.570001274 +0000 UTC m=+7.659528217" watchObservedRunningTime="2026-11-04 07:15:01.565394262 +0000 UTC m=+8.654921208" Nov 04 07:15:01.566243 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:01.566195 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-sd2bs" podStartSLOduration=2.333048668 podStartE2EDuration="7.566179636s" totalImagesPullingTime="5.233130968s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:14:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:01.565260908 +0000 UTC m=+8.654787877" watchObservedRunningTime="2026-11-04 07:15:01.566179636 +0000 UTC m=+8.655706586" Nov 04 07:15:02.429276 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:02.429241 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:02.429491 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:02.429381 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:02.553696 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:02.553604 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" event={"ID":"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c","Type":"ContainerStarted","Data":"0b56490edcc1ed913df258860798023c4a98ed5896b33894f8196d778abfba2f"} Nov 04 07:15:02.979646 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:02.979558 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:02.979833 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:02.979749 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 07:15:02.979833 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:02.979831 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret podName:fa9b645b-69d5-47a7-8457-409364209023 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:10.979810179 +0000 UTC m=+18.069337112 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret") pod "global-pull-secret-syncer-m7gt8" (UID: "fa9b645b-69d5-47a7-8457-409364209023") : object "kube-system"/"original-pull-secret" not registered Nov 04 07:15:03.080373 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:03.080330 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs\") pod \"network-metrics-daemon-ldwvk\" (UID: \"17916760-be6c-47bf-8569-94ec0e36e304\") " pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:03.080373 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:03.080380 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhnsv\" (UniqueName: \"kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv\") pod \"network-check-target-9qgxt\" (UID: \"97707149-0248-45a0-87d2-7347b960c2db\") " pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:03.080647 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:03.080553 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 07:15:03.080647 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:03.080575 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 07:15:03.080647 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:03.080588 2583 projected.go:196] Error preparing data for projected volume kube-api-access-nhnsv for pod openshift-network-diagnostics/network-check-target-9qgxt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 07:15:03.080820 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:03.080650 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv podName:97707149-0248-45a0-87d2-7347b960c2db nodeName:}" failed. No retries permitted until 2026-11-04 07:15:11.080634259 +0000 UTC m=+18.170161178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nhnsv" (UniqueName: "kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv") pod "network-check-target-9qgxt" (UID: "97707149-0248-45a0-87d2-7347b960c2db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 07:15:03.081130 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:03.081062 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 07:15:03.081130 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:03.081109 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs podName:17916760-be6c-47bf-8569-94ec0e36e304 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:11.08109535 +0000 UTC m=+18.170622270 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs") pod "network-metrics-daemon-ldwvk" (UID: "17916760-be6c-47bf-8569-94ec0e36e304") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 07:15:03.432534 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:03.432460 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:03.432722 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:03.432633 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:03.433068 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:03.433042 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:03.433172 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:03.433138 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:04.036940 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:04.036916 2583 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 07:15:04.360215 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:04.360094 2583 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T07:15:04.036938538Z","UUID":"aa01d661-1b2a-423e-b0a5-d05b529abc24","Handler":null,"Name":"","Endpoint":""} Nov 04 07:15:04.362599 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:04.362564 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 07:15:04.362599 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:04.362600 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 07:15:04.429498 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:04.429472 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:04.429630 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:04.429606 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:04.559177 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:04.559141 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-x6v9g" event={"ID":"19b2ab56-a868-4570-b098-37c517da236a","Type":"ContainerStarted","Data":"b48227ec10bbbc8d6e45a6b731d07f33f5029b1dcf1f00949b06e44a9e9f5553"} Nov 04 07:15:04.561517 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:04.561474 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" event={"ID":"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c","Type":"ContainerStarted","Data":"99db86488f546aa9dd5ab95c1bc769a4b7cbdeaabdb664f40723895cc8682de9"} Nov 04 07:15:04.563634 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:04.563604 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grc4v" event={"ID":"80a4a0ad-5738-454b-9818-16b544aa5284","Type":"ContainerStarted","Data":"064eeee9c05f1fc7df16e2133423235f6c14b7e4dd6732a0d09c142432f857a1"} Nov 04 07:15:04.565373 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:04.565350 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6jbrj" event={"ID":"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f","Type":"ContainerStarted","Data":"7bcf7af9727af4c405eb510ce6cfb94ada2d7e4e41eaa9c72d591719c17ed86b"} Nov 04 07:15:04.576433 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:04.576382 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-x6v9g" podStartSLOduration=2.34796697 podStartE2EDuration="10.576368159s" totalImagesPullingTime="8.228401189s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:14:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:04.576029999 +0000 UTC m=+11.665556938" watchObservedRunningTime="2026-11-04 07:15:04.576368159 +0000 UTC m=+11.665895099" Nov 04 07:15:04.596056 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:04.595972 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-6jbrj" podStartSLOduration=2.065300171 podStartE2EDuration="10.595956157s" totalImagesPullingTime="8.530655986s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:14:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:04.595388488 +0000 UTC m=+11.684915429" watchObservedRunningTime="2026-11-04 07:15:04.595956157 +0000 UTC m=+11.685483091" Nov 04 07:15:05.146398 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:05.146143 2583 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2s2xx" Nov 04 07:15:05.147351 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:05.147140 2583 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2s2xx" Nov 04 07:15:05.429835 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:05.429722 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:05.429835 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:05.429739 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:05.430044 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:05.429877 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:05.430307 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:05.430248 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:05.570903 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:05.570869 2583 generic.go:356] "Generic (PLEG): container finished" podID="80a4a0ad-5738-454b-9818-16b544aa5284" containerID="064eeee9c05f1fc7df16e2133423235f6c14b7e4dd6732a0d09c142432f857a1" exitCode=0 Nov 04 07:15:05.571870 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:05.571842 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grc4v" event={"ID":"80a4a0ad-5738-454b-9818-16b544aa5284","Type":"ContainerDied","Data":"064eeee9c05f1fc7df16e2133423235f6c14b7e4dd6732a0d09c142432f857a1"} Nov 04 07:15:06.429543 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:06.429461 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:06.429933 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:06.429579 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:06.574910 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:06.574866 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s8gfp" event={"ID":"63631d09-f1c5-43a6-98e9-4002ada7949a","Type":"ContainerStarted","Data":"c2d9635e73a479321885b0732b41089b805089d0e64411ba1c528404dda2322a"} Nov 04 07:15:06.577189 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:06.577159 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" event={"ID":"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c","Type":"ContainerStarted","Data":"9ec54c9d05a4e693112de370ec1467b3ee1284560af2137a75513e70477d01bf"} Nov 04 07:15:06.592714 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:06.592671 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-s8gfp" podStartSLOduration=3.90110015 podStartE2EDuration="12.592656774s" totalImagesPullingTime="8.691556624s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:14:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:06.592043926 +0000 UTC m=+13.681570868" watchObservedRunningTime="2026-11-04 07:15:06.592656774 +0000 UTC m=+13.682183713" Nov 04 07:15:06.611919 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:06.611879 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" podStartSLOduration=3.154419444 podStartE2EDuration="12.611863247s" totalImagesPullingTime="9.457443803s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:14:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:06.611598932 +0000 UTC m=+13.701125870" watchObservedRunningTime="2026-11-04 07:15:06.611863247 +0000 UTC m=+13.701390187" Nov 04 07:15:07.429093 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:07.429062 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:07.429263 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:07.429104 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:07.429263 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:07.429188 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:07.429391 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:07.429326 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:07.581023 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:07.580986 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" event={"ID":"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c","Type":"ContainerStarted","Data":"b29c3f94a4dde73b9a34981ceb224a3e049be8052ccc59ae419e893c6ca2a982"} Nov 04 07:15:07.583963 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:07.583934 2583 generic.go:356] "Generic (PLEG): container finished" podID="80a4a0ad-5738-454b-9818-16b544aa5284" containerID="283569bdc3646cf491be01540a163c5e4a380943789d0fc2d26dae67134f6f4e" exitCode=0 Nov 04 07:15:07.584099 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:07.584015 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grc4v" event={"ID":"80a4a0ad-5738-454b-9818-16b544aa5284","Type":"ContainerDied","Data":"283569bdc3646cf491be01540a163c5e4a380943789d0fc2d26dae67134f6f4e"} Nov 04 07:15:07.601719 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:07.601678 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" podStartSLOduration=2.611788053 podStartE2EDuration="13.601655073s" totalImagesPullingTime="10.98986702s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:14:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:07.601539993 +0000 UTC m=+14.691066932" watchObservedRunningTime="2026-11-04 07:15:07.601655073 +0000 UTC m=+14.691182012" Nov 04 07:15:08.429517 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:08.429485 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:08.429669 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:08.429626 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:08.588753 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:08.588673 2583 generic.go:356] "Generic (PLEG): container finished" podID="80a4a0ad-5738-454b-9818-16b544aa5284" containerID="cf1d9f277fec1a4d634e1a8e2521c9a60f2d349a276a1e17f2f24b16ec67d52a" exitCode=0 Nov 04 07:15:08.589594 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:08.588756 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grc4v" event={"ID":"80a4a0ad-5738-454b-9818-16b544aa5284","Type":"ContainerDied","Data":"cf1d9f277fec1a4d634e1a8e2521c9a60f2d349a276a1e17f2f24b16ec67d52a"} Nov 04 07:15:09.429151 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:09.429113 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:09.429331 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:09.429159 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:09.429331 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:09.429278 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:09.429439 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:09.429402 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:10.429188 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:10.429156 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:10.429620 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:10.429263 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:10.595288 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:10.595001 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grc4v" event={"ID":"80a4a0ad-5738-454b-9818-16b544aa5284","Type":"ContainerStarted","Data":"cbe8dad75dfc4571687cbff1f83d8654f69ce08cdb4be70dedccdda342525b72"} Nov 04 07:15:11.044596 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:11.044568 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:11.044691 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:11.044677 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 07:15:11.044739 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:11.044730 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret podName:fa9b645b-69d5-47a7-8457-409364209023 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:27.044716221 +0000 UTC m=+34.134243152 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret") pod "global-pull-secret-syncer-m7gt8" (UID: "fa9b645b-69d5-47a7-8457-409364209023") : object "kube-system"/"original-pull-secret" not registered Nov 04 07:15:11.145409 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:11.145371 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs\") pod \"network-metrics-daemon-ldwvk\" (UID: \"17916760-be6c-47bf-8569-94ec0e36e304\") " pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:11.145597 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:11.145419 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhnsv\" (UniqueName: \"kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv\") pod \"network-check-target-9qgxt\" (UID: \"97707149-0248-45a0-87d2-7347b960c2db\") " pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:11.145597 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:11.145536 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 07:15:11.145726 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:11.145603 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs podName:17916760-be6c-47bf-8569-94ec0e36e304 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:27.145589731 +0000 UTC m=+34.235116665 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs") pod "network-metrics-daemon-ldwvk" (UID: "17916760-be6c-47bf-8569-94ec0e36e304") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 07:15:11.145726 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:11.145610 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 07:15:11.145726 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:11.145628 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 07:15:11.145726 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:11.145640 2583 projected.go:196] Error preparing data for projected volume kube-api-access-nhnsv for pod openshift-network-diagnostics/network-check-target-9qgxt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 07:15:11.145726 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:11.145694 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv podName:97707149-0248-45a0-87d2-7347b960c2db nodeName:}" failed. No retries permitted until 2026-11-04 07:15:27.145678692 +0000 UTC m=+34.235205622 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nhnsv" (UniqueName: "kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv") pod "network-check-target-9qgxt" (UID: "97707149-0248-45a0-87d2-7347b960c2db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 07:15:11.429617 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:11.429424 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:11.430053 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:11.429489 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:11.430053 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:11.429696 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:11.430053 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:11.429780 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:11.599425 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:11.599380 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" event={"ID":"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad","Type":"ContainerStarted","Data":"c319fa873c2245f1cad8c66fd3cfd5d62f92c7e1fa116e60d3609fc0ecf0853e"} Nov 04 07:15:11.599616 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:11.599458 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" event={"ID":"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad","Type":"ContainerStarted","Data":"688f094e45f258c9249b7ab33279edbf678a58fa61de4208e389f3ad9f21087e"} Nov 04 07:15:11.599616 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:11.599489 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" event={"ID":"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad","Type":"ContainerStarted","Data":"2e6c295e055d4b0b6ef067cd86aac0c2b22111079f2d88996780b6a35d95e296"} Nov 04 07:15:11.599616 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:11.599540 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" event={"ID":"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad","Type":"ContainerStarted","Data":"c19595a69af112cf50dea1590960f8e5d46f44939f6c4ca09edab8f6bb7496e9"} Nov 04 07:15:11.599616 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:11.599568 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" event={"ID":"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad","Type":"ContainerStarted","Data":"4db08208f794775025a0390a21295e5014c2e8ef2c087b47c899688df21dc32a"} Nov 04 07:15:11.599616 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:11.599600 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" event={"ID":"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad","Type":"ContainerStarted","Data":"e7504f85c8e0ea0d2ee2d38d2211ff402aff22891e66313553de9a14e1c771a9"} Nov 04 07:15:11.601971 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:11.601946 2583 generic.go:356] "Generic (PLEG): container finished" podID="80a4a0ad-5738-454b-9818-16b544aa5284" containerID="cbe8dad75dfc4571687cbff1f83d8654f69ce08cdb4be70dedccdda342525b72" exitCode=0 Nov 04 07:15:11.602062 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:11.601987 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grc4v" event={"ID":"80a4a0ad-5738-454b-9818-16b544aa5284","Type":"ContainerDied","Data":"cbe8dad75dfc4571687cbff1f83d8654f69ce08cdb4be70dedccdda342525b72"} Nov 04 07:15:12.429180 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:12.429140 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:12.429390 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:12.429266 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:12.607311 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:12.607279 2583 generic.go:356] "Generic (PLEG): container finished" podID="80a4a0ad-5738-454b-9818-16b544aa5284" containerID="824f39ef40be88347624d0f709aa880743eb5f54a48113fcc7854259aed4cb3f" exitCode=0 Nov 04 07:15:12.607311 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:12.607319 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grc4v" event={"ID":"80a4a0ad-5738-454b-9818-16b544aa5284","Type":"ContainerDied","Data":"824f39ef40be88347624d0f709aa880743eb5f54a48113fcc7854259aed4cb3f"} Nov 04 07:15:13.429980 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:13.429946 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:13.430177 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:13.430113 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:13.430177 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:13.430150 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:13.430298 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:13.430216 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:13.612266 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:13.612228 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-grc4v" event={"ID":"80a4a0ad-5738-454b-9818-16b544aa5284","Type":"ContainerStarted","Data":"7280525eb8d2c0a013914f1666f3d1f6e437adb3c9c5552f550687ef4d7e782c"} Nov 04 07:15:13.615398 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:13.615375 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" event={"ID":"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad","Type":"ContainerStarted","Data":"0a8ce02904274253c6bf8194a614e0885141ed344e4edd0649154d35d1f84e5c"} Nov 04 07:15:14.230565 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:14.230533 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:14.396754 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:14.396724 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:14.429816 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:14.429792 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:14.429962 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:14.429882 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:15.265394 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:15.265360 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:15.359178 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:15.358985 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:15.429835 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:15.429760 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:15.429835 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:15.429796 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:15.430012 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:15.429894 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:15.430012 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:15.429984 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:15.622894 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:15.622862 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" event={"ID":"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad","Type":"ContainerStarted","Data":"3a9badc728db452241093765a2df8b5252f2e1c90f67fa787442e6b58121fa85"} Nov 04 07:15:15.623114 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:15.623097 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:15:15.638448 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:15.638427 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:15:15.657804 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:15.657769 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-grc4v" podStartSLOduration=10.178533756 podStartE2EDuration="21.657757722s" totalImagesPullingTime="11.479223966s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:14:54 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:13.64587631 +0000 UTC m=+20.735403248" watchObservedRunningTime="2026-11-04 07:15:15.657757722 +0000 UTC m=+22.747284662" Nov 04 07:15:15.658138 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:15.658113 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" podStartSLOduration=6.607858366 podStartE2EDuration="21.658108767s" totalImagesPullingTime="15.050250401s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:14:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:15.656036675 +0000 UTC m=+22.745563613" watchObservedRunningTime="2026-11-04 07:15:15.658108767 +0000 UTC m=+22.747635741" Nov 04 07:15:16.429824 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:16.429787 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:16.430466 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:16.430442 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:16.629853 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:16.629818 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:15:16.630002 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:16.629959 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:15:16.644245 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:16.644215 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:15:16.870800 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:16.870765 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9qgxt"] Nov 04 07:15:16.871001 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:16.870903 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:16.871052 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:16.871012 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:16.877349 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:16.877318 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ldwvk"] Nov 04 07:15:16.877482 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:16.877421 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:16.877556 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:16.877535 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:16.877920 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:16.877898 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-m7gt8"] Nov 04 07:15:16.878006 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:16.877996 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:16.878101 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:16.878084 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:18.147320 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:18.147288 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:18.240874 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:18.240842 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:18.429197 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:18.429118 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:18.429335 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:18.429233 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:18.429335 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:18.429248 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:18.429408 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:18.429351 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:18.429408 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:18.429240 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:18.429468 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:18.429452 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:19.109679 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:19.109646 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:19.208629 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:19.208595 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:19.727326 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:19.727299 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:19.825823 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:19.825785 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:19.930808 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:19.930773 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:20.429040 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:20.429009 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:20.429496 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:20.429048 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:20.429496 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:20.429124 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:20.429496 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:20.429177 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:20.429496 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:20.429264 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:20.429496 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:20.429341 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:20.812812 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:20.812700 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:20.962282 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:20.962241 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:21.144844 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:21.144813 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:21.306768 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:21.306736 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2s2xx" Nov 04 07:15:21.306933 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:21.306876 2583 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 07:15:21.307395 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:21.307373 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2s2xx" Nov 04 07:15:21.397444 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:21.397344 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:21.518075 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:21.518040 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:21.616128 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:21.616095 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:22.312705 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:22.312635 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:22.429734 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:22.429708 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:22.429879 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:22.429749 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:22.429879 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:22.429858 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:22.429958 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:22.429903 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:22.429958 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:22.429924 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:22.430035 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:22.429991 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:22.727072 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:22.727023 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:22.826678 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:22.826613 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:22.946587 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:22.946561 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:23.050216 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:23.050143 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:23.218236 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:23.218202 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:23.853710 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:23.853668 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:24.043331 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:24.043159 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:24.144085 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:24.144055 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:24.429321 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:24.429235 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:24.429321 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:24.429277 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:24.429536 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:24.429362 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:24.429536 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:24.429410 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:24.429536 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:24.429492 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:24.429678 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:24.429589 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:24.435395 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:24.435347 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:24.537854 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:24.537813 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:24.634567 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:24.634535 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:24.834932 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:24.834850 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:24.934748 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:24.934711 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:25.031222 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:25.031194 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:25.179522 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:25.179489 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:25.922973 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:25.922950 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-x6v9g_19b2ab56-a868-4570-b098-37c517da236a/dns-node-resolver/0.log" Nov 04 07:15:25.954068 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:25.954018 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:26.390724 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:26.390691 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:26.429933 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:26.429904 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:26.430072 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:26.429999 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:26.430072 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:26.430006 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:26.430072 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:26.430024 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:26.430165 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:26.430101 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:26.430198 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:26.430164 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:26.595130 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:26.595093 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:26.782556 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:26.782467 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:26.922352 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:26.922328 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sd2bs_f33f6c60-eff2-40c2-9350-ba10eaf5442d/node-ca/0.log" Nov 04 07:15:26.987203 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:26.987173 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:27.058472 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:27.058413 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:27.058584 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:27.058538 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 07:15:27.058584 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:27.058583 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret podName:fa9b645b-69d5-47a7-8457-409364209023 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:59.058571252 +0000 UTC m=+66.148098173 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret") pod "global-pull-secret-syncer-m7gt8" (UID: "fa9b645b-69d5-47a7-8457-409364209023") : object "kube-system"/"original-pull-secret" not registered Nov 04 07:15:27.089259 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:27.089234 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:27.159077 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:27.159050 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs\") pod \"network-metrics-daemon-ldwvk\" (UID: \"17916760-be6c-47bf-8569-94ec0e36e304\") " pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:27.159182 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:27.159080 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhnsv\" (UniqueName: \"kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv\") pod \"network-check-target-9qgxt\" (UID: \"97707149-0248-45a0-87d2-7347b960c2db\") " pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:27.159234 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:27.159182 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 07:15:27.159234 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:27.159193 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 07:15:27.159234 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:27.159205 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 07:15:27.159234 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:27.159213 2583 projected.go:196] Error preparing data for projected volume kube-api-access-nhnsv for pod openshift-network-diagnostics/network-check-target-9qgxt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 07:15:27.159234 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:27.159230 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs podName:17916760-be6c-47bf-8569-94ec0e36e304 nodeName:}" failed. No retries permitted until 2026-11-04 07:15:59.159218576 +0000 UTC m=+66.248745493 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs") pod "network-metrics-daemon-ldwvk" (UID: "17916760-be6c-47bf-8569-94ec0e36e304") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 07:15:27.159400 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:27.159245 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv podName:97707149-0248-45a0-87d2-7347b960c2db nodeName:}" failed. No retries permitted until 2026-11-04 07:15:59.159235723 +0000 UTC m=+66.248762663 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nhnsv" (UniqueName: "kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv") pod "network-check-target-9qgxt" (UID: "97707149-0248-45a0-87d2-7347b960c2db") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 07:15:27.206416 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:27.206391 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:28.391066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:28.391029 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:28.429855 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:28.429827 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:28.429983 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:28.429906 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:28.430025 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:28.429987 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:28.430025 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:28.429994 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:28.430086 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:28.430054 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:28.430142 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:28.430120 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:28.581962 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:28.581933 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:28.779426 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:28.779351 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:28.879658 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:28.879625 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:28.980137 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:28.980107 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:29.184142 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:29.184111 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:30.185310 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:30.185277 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:30.429651 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:30.429622 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:30.429813 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:30.429626 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:30.429813 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:30.429731 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:30.429813 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:30.429626 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:30.429813 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:30.429798 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:30.429939 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:30.429860 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:30.621812 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:30.621782 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:30.787945 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:30.787916 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:30.892758 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:30.892690 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:30.999116 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:30.999087 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:31.098839 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:31.098813 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:31.376452 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:31.376424 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:31.989440 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:31.989395 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:32.177594 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:32.177563 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:32.429777 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:32.429750 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:32.430191 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:32.429853 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:32.430191 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:32.429859 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:32.430191 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:32.429857 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:32.430191 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:32.429934 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:32.430191 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:32.429995 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:32.579413 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:32.579369 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:32.790214 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:32.790137 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:33.003702 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:33.003671 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:33.335141 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:33.335112 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:33.435672 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:33.435639 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:33.582991 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:33.582964 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:34.429522 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:34.429479 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:34.429522 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:34.429525 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:34.429729 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:34.429528 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:34.429729 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:34.429588 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:34.429729 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:34.429656 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:34.429729 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:34.429712 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:36.429489 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:36.429460 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:36.429983 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:36.429596 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:36.429983 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:36.429610 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:36.429983 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:36.429686 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:36.429983 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:36.429733 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:36.429983 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:36.429804 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:38.429697 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:38.429670 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:38.430077 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:38.429668 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:38.430077 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:38.429770 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:38.430077 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:38.429669 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:38.430077 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:38.429838 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:38.430077 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:38.429918 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:38.441114 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:38.441089 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:38.553286 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:38.553255 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:38.659333 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:38.659298 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:38.790110 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:38.790042 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:38.904952 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:38.904926 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:39.692340 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:39.692311 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:40.429084 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:40.429059 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:40.429084 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:40.429079 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:40.429295 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:40.429058 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:40.429295 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:40.429151 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:40.429295 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:40.429224 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:40.429390 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:40.429297 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:42.429722 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:42.429689 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:42.429722 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:42.429726 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:42.429722 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:42.429724 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:42.430175 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:42.429796 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:42.430175 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:42.429855 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:42.430175 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:42.429915 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:43.758253 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:43.758220 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:43.938884 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:43.938856 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:44.130602 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.130568 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:44.247408 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.247379 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:44.429825 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.429740 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:44.429965 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.429741 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:44.429965 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:44.429853 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ldwvk" podUID="17916760-be6c-47bf-8569-94ec0e36e304" Nov 04 07:15:44.429965 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.429740 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:44.429965 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:44.429928 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-m7gt8" podUID="fa9b645b-69d5-47a7-8457-409364209023" Nov 04 07:15:44.430089 ip-10-0-131-6 kubenswrapper[2583]: E1104 07:15:44.429994 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9qgxt" podUID="97707149-0248-45a0-87d2-7347b960c2db" Nov 04 07:15:44.746125 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.746054 2583 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeReady" Nov 04 07:15:44.746256 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.746174 2583 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Nov 04 07:15:44.806378 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.806336 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qpt88"] Nov 04 07:15:44.808105 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.808089 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qpt88" Nov 04 07:15:44.810243 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.810218 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-g6lhj"] Nov 04 07:15:44.811753 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.811736 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:15:44.813363 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.813322 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6pjn4\"" Nov 04 07:15:44.813480 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.813439 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 04 07:15:44.814320 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.814303 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 04 07:15:44.821341 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.821317 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qpt88"] Nov 04 07:15:44.824457 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.824436 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g6lhj"] Nov 04 07:15:44.889221 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.889192 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:15:44.889399 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.889229 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-crio-socket\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:15:44.889399 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.889245 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-data-volume\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:15:44.889399 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.889267 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:15:44.889399 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.889394 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/508fee92-08e3-49ab-8b16-3add538b11eb-metrics-tls\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:15:44.889615 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.889417 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/508fee92-08e3-49ab-8b16-3add538b11eb-config-volume\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:15:44.889615 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.889442 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4rg8\" (UniqueName: \"kubernetes.io/projected/508fee92-08e3-49ab-8b16-3add538b11eb-kube-api-access-j4rg8\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:15:44.889615 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.889460 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jmmp\" (UniqueName: \"kubernetes.io/projected/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-kube-api-access-5jmmp\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:15:44.889615 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.889538 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/508fee92-08e3-49ab-8b16-3add538b11eb-tmp-dir\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:15:44.912039 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.912010 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pksw6"] Nov 04 07:15:44.912974 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.912949 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9lks7\"" Nov 04 07:15:44.913066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.912958 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 04 07:15:44.913778 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.913758 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 04 07:15:44.913778 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.913761 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pksw6" Nov 04 07:15:44.913900 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.913827 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 04 07:15:44.914106 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.914090 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 04 07:15:44.925417 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.925399 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pksw6"] Nov 04 07:15:44.990441 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.990409 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4rg8\" (UniqueName: \"kubernetes.io/projected/508fee92-08e3-49ab-8b16-3add538b11eb-kube-api-access-j4rg8\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:15:44.990639 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.990448 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jmmp\" (UniqueName: \"kubernetes.io/projected/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-kube-api-access-5jmmp\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:15:44.990639 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.990468 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/508fee92-08e3-49ab-8b16-3add538b11eb-tmp-dir\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:15:44.990639 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.990606 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:15:44.990805 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.990645 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-crio-socket\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:15:44.990805 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.990666 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-data-volume\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:15:44.990805 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.990684 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:15:44.990805 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.990703 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d085676f-b286-4834-9e62-039f2b6e7f72-cert\") pod \"ingress-canary-pksw6\" (UID: \"d085676f-b286-4834-9e62-039f2b6e7f72\") " pod="openshift-ingress-canary/ingress-canary-pksw6" Nov 04 07:15:44.990805 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.990778 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/508fee92-08e3-49ab-8b16-3add538b11eb-metrics-tls\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:15:44.991027 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.990881 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/508fee92-08e3-49ab-8b16-3add538b11eb-config-volume\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:15:44.991027 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.990919 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2wdv\" (UniqueName: \"kubernetes.io/projected/d085676f-b286-4834-9e62-039f2b6e7f72-kube-api-access-x2wdv\") pod \"ingress-canary-pksw6\" (UID: \"d085676f-b286-4834-9e62-039f2b6e7f72\") " pod="openshift-ingress-canary/ingress-canary-pksw6" Nov 04 07:15:44.991027 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.990952 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-crio-socket\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:15:44.991027 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.990982 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/508fee92-08e3-49ab-8b16-3add538b11eb-tmp-dir\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:15:44.991206 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.991098 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-data-volume\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:15:44.991344 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.991322 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:15:44.991408 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.991374 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/508fee92-08e3-49ab-8b16-3add538b11eb-config-volume\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:15:44.994408 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.994378 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/508fee92-08e3-49ab-8b16-3add538b11eb-metrics-tls\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:15:44.994408 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:44.994391 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:15:45.007110 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.007051 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4rg8\" (UniqueName: \"kubernetes.io/projected/508fee92-08e3-49ab-8b16-3add538b11eb-kube-api-access-j4rg8\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:15:45.007219 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.007164 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jmmp\" (UniqueName: \"kubernetes.io/projected/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-kube-api-access-5jmmp\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:15:45.013847 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.013826 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 04 07:15:45.013978 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.013963 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-7rfcz\"" Nov 04 07:15:45.014026 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.014004 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 04 07:15:45.014070 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.014024 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 04 07:15:45.091382 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.091344 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2wdv\" (UniqueName: \"kubernetes.io/projected/d085676f-b286-4834-9e62-039f2b6e7f72-kube-api-access-x2wdv\") pod \"ingress-canary-pksw6\" (UID: \"d085676f-b286-4834-9e62-039f2b6e7f72\") " pod="openshift-ingress-canary/ingress-canary-pksw6" Nov 04 07:15:45.091560 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.091544 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d085676f-b286-4834-9e62-039f2b6e7f72-cert\") pod \"ingress-canary-pksw6\" (UID: \"d085676f-b286-4834-9e62-039f2b6e7f72\") " pod="openshift-ingress-canary/ingress-canary-pksw6" Nov 04 07:15:45.093840 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.093821 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d085676f-b286-4834-9e62-039f2b6e7f72-cert\") pod \"ingress-canary-pksw6\" (UID: \"d085676f-b286-4834-9e62-039f2b6e7f72\") " pod="openshift-ingress-canary/ingress-canary-pksw6" Nov 04 07:15:45.104000 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.103975 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2wdv\" (UniqueName: \"kubernetes.io/projected/d085676f-b286-4834-9e62-039f2b6e7f72-kube-api-access-x2wdv\") pod \"ingress-canary-pksw6\" (UID: \"d085676f-b286-4834-9e62-039f2b6e7f72\") " pod="openshift-ingress-canary/ingress-canary-pksw6" Nov 04 07:15:45.121851 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.121828 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qpt88" Nov 04 07:15:45.128288 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.128254 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:15:45.223112 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.223080 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pksw6" Nov 04 07:15:45.413041 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.413007 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-g6lhj"] Nov 04 07:15:45.413671 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.413635 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qpt88"] Nov 04 07:15:45.417581 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:15:45.417553 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod508fee92_08e3_49ab_8b16_3add538b11eb.slice/crio-b863cc9011c198aadf7cc1bf200574267b9da8b94ddfc802cd35e134a423fd52 WatchSource:0}: Error finding container b863cc9011c198aadf7cc1bf200574267b9da8b94ddfc802cd35e134a423fd52: Status 404 returned error can't find the container with id b863cc9011c198aadf7cc1bf200574267b9da8b94ddfc802cd35e134a423fd52 Nov 04 07:15:45.491591 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.491562 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pksw6"] Nov 04 07:15:45.495644 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:15:45.495623 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd085676f_b286_4834_9e62_039f2b6e7f72.slice/crio-e1efd0dbd7ffad01b836b1df99461789e63e533d166951a6420a7a5596cb62bf WatchSource:0}: Error finding container e1efd0dbd7ffad01b836b1df99461789e63e533d166951a6420a7a5596cb62bf: Status 404 returned error can't find the container with id e1efd0dbd7ffad01b836b1df99461789e63e533d166951a6420a7a5596cb62bf Nov 04 07:15:45.682222 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.682190 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qpt88" event={"ID":"508fee92-08e3-49ab-8b16-3add538b11eb","Type":"ContainerStarted","Data":"b863cc9011c198aadf7cc1bf200574267b9da8b94ddfc802cd35e134a423fd52"} Nov 04 07:15:45.683320 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.683299 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pksw6" event={"ID":"d085676f-b286-4834-9e62-039f2b6e7f72","Type":"ContainerStarted","Data":"e1efd0dbd7ffad01b836b1df99461789e63e533d166951a6420a7a5596cb62bf"} Nov 04 07:15:45.684682 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.684664 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g6lhj" event={"ID":"7dcd4bbc-400c-4fce-b87c-0c018fadecfa","Type":"ContainerStarted","Data":"deca6a3cd41c2fd3355c567e024a0862136b4b8f3be069d722214a95d9c2a17a"} Nov 04 07:15:45.684774 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:45.684700 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g6lhj" event={"ID":"7dcd4bbc-400c-4fce-b87c-0c018fadecfa","Type":"ContainerStarted","Data":"684c4196729a5a5c95e2abb153f5df3b2a2c0f984ecc4c757a346a9f663b5e7a"} Nov 04 07:15:46.429844 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:46.429805 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:46.430301 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:46.430238 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:46.430541 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:46.430521 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:46.440963 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:46.440945 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 07:15:46.441065 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:46.441038 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-zw52p\"" Nov 04 07:15:46.442566 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:46.442538 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-hvg8n\"" Nov 04 07:15:46.442711 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:46.442693 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 07:15:46.442817 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:46.442805 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 07:15:46.443811 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:46.443790 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gnthh\"" Nov 04 07:15:46.444475 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:46.444452 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 07:15:47.078288 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.078204 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:47.107936 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.107907 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5"] Nov 04 07:15:47.110043 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.110021 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:15:47.117377 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.117330 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Nov 04 07:15:47.118370 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.118316 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 04 07:15:47.118476 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.118436 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 04 07:15:47.118545 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.118527 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-8zskt\"" Nov 04 07:15:47.118597 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.118558 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Nov 04 07:15:47.118649 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.118610 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 04 07:15:47.121060 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.121022 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5"] Nov 04 07:15:47.178077 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.177990 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:47.207003 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.206964 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/51538327-34ef-409b-b76a-6963b7254eec-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:15:47.207205 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.207028 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m7f6\" (UniqueName: \"kubernetes.io/projected/51538327-34ef-409b-b76a-6963b7254eec-kube-api-access-7m7f6\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:15:47.207205 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.207099 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/51538327-34ef-409b-b76a-6963b7254eec-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:15:47.207205 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.207126 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51538327-34ef-409b-b76a-6963b7254eec-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:15:47.304229 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.304122 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:47.308307 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.307483 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/51538327-34ef-409b-b76a-6963b7254eec-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:15:47.308307 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.307540 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51538327-34ef-409b-b76a-6963b7254eec-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:15:47.308307 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.307628 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/51538327-34ef-409b-b76a-6963b7254eec-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:15:47.308307 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.307673 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m7f6\" (UniqueName: \"kubernetes.io/projected/51538327-34ef-409b-b76a-6963b7254eec-kube-api-access-7m7f6\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:15:47.309637 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.309141 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51538327-34ef-409b-b76a-6963b7254eec-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:15:47.311587 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.311539 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/51538327-34ef-409b-b76a-6963b7254eec-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:15:47.316063 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.316011 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/51538327-34ef-409b-b76a-6963b7254eec-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:15:47.330384 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.330354 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m7f6\" (UniqueName: \"kubernetes.io/projected/51538327-34ef-409b-b76a-6963b7254eec-kube-api-access-7m7f6\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:15:47.428473 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.428435 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:15:47.694386 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.694345 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qpt88" event={"ID":"508fee92-08e3-49ab-8b16-3add538b11eb","Type":"ContainerStarted","Data":"dd08557568546047cb31aa91a30f21db3234f1a4de4bc0569a6ce3830c89325d"} Nov 04 07:15:47.694861 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.694411 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qpt88" event={"ID":"508fee92-08e3-49ab-8b16-3add538b11eb","Type":"ContainerStarted","Data":"efae8a6369d1e23caa4906304ffd1dce918ae6e5c5d5d82511caa0b49ac245ab"} Nov 04 07:15:47.694861 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.694579 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qpt88" Nov 04 07:15:47.696702 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.696679 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g6lhj" event={"ID":"7dcd4bbc-400c-4fce-b87c-0c018fadecfa","Type":"ContainerStarted","Data":"7f475839ffca82a1f6864984033a91d0b9a58b35c62a60018320a33b2d66c764"} Nov 04 07:15:47.709655 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.709625 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5"] Nov 04 07:15:47.713544 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:15:47.713517 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51538327_34ef_409b_b76a_6963b7254eec.slice/crio-4f9d24ff5fa1251fad621af3e696722792385fda408c20e827e43db5e133ec3b WatchSource:0}: Error finding container 4f9d24ff5fa1251fad621af3e696722792385fda408c20e827e43db5e133ec3b: Status 404 returned error can't find the container with id 4f9d24ff5fa1251fad621af3e696722792385fda408c20e827e43db5e133ec3b Nov 04 07:15:47.714169 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:47.714119 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-qpt88" podStartSLOduration=1.968946693 podStartE2EDuration="3.714104239s" totalImagesPullingTime="1.745157546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:47.713689822 +0000 UTC m=+54.803216779" watchObservedRunningTime="2026-11-04 07:15:47.714104239 +0000 UTC m=+54.803631182" Nov 04 07:15:48.642695 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:48.642668 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:15:48.701933 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:48.701894 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-g6lhj" event={"ID":"7dcd4bbc-400c-4fce-b87c-0c018fadecfa","Type":"ContainerStarted","Data":"a4fa8c58d092589b6910e93a85633a537d2e0f7b971ca78f6711732fec3d6895"} Nov 04 07:15:48.703405 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:48.703373 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" event={"ID":"51538327-34ef-409b-b76a-6963b7254eec","Type":"ContainerStarted","Data":"4f9d24ff5fa1251fad621af3e696722792385fda408c20e827e43db5e133ec3b"} Nov 04 07:15:48.705045 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:48.705015 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pksw6" event={"ID":"d085676f-b286-4834-9e62-039f2b6e7f72","Type":"ContainerStarted","Data":"f031fc23b1c7e647863655b32fdc2dcbcb8008582aa79ce28a7f017662339a69"} Nov 04 07:15:48.724673 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:48.724622 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-g6lhj" podStartSLOduration=1.9791402900000001 podStartE2EDuration="4.724608619s" totalImagesPullingTime="2.745468329s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:15:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:48.723563066 +0000 UTC m=+55.813090024" watchObservedRunningTime="2026-11-04 07:15:48.724608619 +0000 UTC m=+55.814135557" Nov 04 07:15:48.743466 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:48.743418 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pksw6" podStartSLOduration=1.9455104890000001 podStartE2EDuration="4.743404044s" totalImagesPullingTime="2.797893555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:48.743106279 +0000 UTC m=+55.832633220" watchObservedRunningTime="2026-11-04 07:15:48.743404044 +0000 UTC m=+55.832930983" Nov 04 07:15:48.813096 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:48.813070 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:48.913642 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:48.913565 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:49.234845 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.234764 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:49.282128 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.282099 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-8656d77c48-qkjzp"] Nov 04 07:15:49.285050 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.285035 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.288945 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.288925 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Nov 04 07:15:49.289076 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.289056 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Nov 04 07:15:49.289342 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.289327 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-h4fsf\"" Nov 04 07:15:49.290383 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.290366 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Nov 04 07:15:49.290467 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.290424 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Nov 04 07:15:49.290467 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.290436 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Nov 04 07:15:49.290582 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.290466 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Nov 04 07:15:49.290582 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.290471 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Nov 04 07:15:49.297818 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.297795 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8656d77c48-qkjzp"] Nov 04 07:15:49.325721 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.325696 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-serving-cert\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.325721 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.325723 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-service-ca\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.325868 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.325745 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1172079a-2931-4ecb-9196-68e6da14fd36-tmp\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.325868 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.325776 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-oauth-serving-cert\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.325948 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.325900 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-oauth-config\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.325948 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.325933 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-console-config\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.326018 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.325950 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hskwv\" (UniqueName: \"kubernetes.io/projected/1172079a-2931-4ecb-9196-68e6da14fd36-kube-api-access-hskwv\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.426930 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.426898 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1172079a-2931-4ecb-9196-68e6da14fd36-tmp\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.427074 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.426946 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-oauth-serving-cert\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.427074 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.427004 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-oauth-config\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.427074 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.427035 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-console-config\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.427074 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.427059 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hskwv\" (UniqueName: \"kubernetes.io/projected/1172079a-2931-4ecb-9196-68e6da14fd36-kube-api-access-hskwv\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.427257 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.427084 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-serving-cert\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.427257 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.427106 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-service-ca\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.427374 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.427350 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1172079a-2931-4ecb-9196-68e6da14fd36-tmp\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.427770 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.427740 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-oauth-serving-cert\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.427872 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.427804 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-console-config\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.427930 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.427910 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-service-ca\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.429653 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.429626 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-oauth-config\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.429736 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.429652 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-serving-cert\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.439453 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.439429 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hskwv\" (UniqueName: \"kubernetes.io/projected/1172079a-2931-4ecb-9196-68e6da14fd36-kube-api-access-hskwv\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.596261 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.596184 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:49.709439 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.709404 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" event={"ID":"51538327-34ef-409b-b76a-6963b7254eec","Type":"ContainerStarted","Data":"000a7b962d3d5e59311d100f09b5d50ddd80c8a47765b3ee2e84b382d924361a"} Nov 04 07:15:49.709818 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.709468 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" event={"ID":"51538327-34ef-409b-b76a-6963b7254eec","Type":"ContainerStarted","Data":"fdb9dff6cb09dcc256cc4e8dfe14a618c181b6c4500e6c8432dffa7ad407449e"} Nov 04 07:15:49.740537 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.740284 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" podStartSLOduration=1.540812648 podStartE2EDuration="2.740268471s" totalImagesPullingTime="1.199455823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:15:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:49.739851742 +0000 UTC m=+56.829378681" watchObservedRunningTime="2026-11-04 07:15:49.740268471 +0000 UTC m=+56.829795411" Nov 04 07:15:49.901598 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:49.901565 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8656d77c48-qkjzp"] Nov 04 07:15:49.904720 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:15:49.904695 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1172079a_2931_4ecb_9196_68e6da14fd36.slice/crio-fc16b866b67f1e6c70bb4ee91fdc2a6fb1b7120131aaaee82fb4f1b3a30bcc0d WatchSource:0}: Error finding container fc16b866b67f1e6c70bb4ee91fdc2a6fb1b7120131aaaee82fb4f1b3a30bcc0d: Status 404 returned error can't find the container with id fc16b866b67f1e6c70bb4ee91fdc2a6fb1b7120131aaaee82fb4f1b3a30bcc0d Nov 04 07:15:50.452383 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:50.452345 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:50.548593 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:50.548534 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:50.715146 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:50.715061 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8656d77c48-qkjzp" event={"ID":"1172079a-2931-4ecb-9196-68e6da14fd36","Type":"ContainerStarted","Data":"fc16b866b67f1e6c70bb4ee91fdc2a6fb1b7120131aaaee82fb4f1b3a30bcc0d"} Nov 04 07:15:51.338871 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.338792 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:51.470146 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.470095 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:51.605798 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.605723 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ktbsj"] Nov 04 07:15:51.608989 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.608964 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.645813 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.645778 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-sys\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.645954 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.645833 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-textfile\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.645954 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.645857 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlj2h\" (UniqueName: \"kubernetes.io/projected/1488becb-97bc-43e6-b85c-6c9c62dfe903-kube-api-access-wlj2h\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.645954 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.645927 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1488becb-97bc-43e6-b85c-6c9c62dfe903-metrics-client-ca\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.646056 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.646024 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-root\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.646056 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.646051 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-accelerators-collector-config\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.646142 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.646081 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-wtmp\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.646210 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.646194 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.646252 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.646224 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-tls\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.647433 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.647410 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:51.666876 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.666847 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Nov 04 07:15:51.667289 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.667268 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Nov 04 07:15:51.667579 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.667566 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l2xtx\"" Nov 04 07:15:51.667773 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.667752 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Nov 04 07:15:51.747420 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.747384 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-textfile\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.747420 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.747421 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlj2h\" (UniqueName: \"kubernetes.io/projected/1488becb-97bc-43e6-b85c-6c9c62dfe903-kube-api-access-wlj2h\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.747973 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.747460 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1488becb-97bc-43e6-b85c-6c9c62dfe903-metrics-client-ca\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.747973 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.747652 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-root\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.747973 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.747705 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-accelerators-collector-config\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.747973 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.747741 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-wtmp\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.747973 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.747795 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.747973 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.747803 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-textfile\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.747973 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.747861 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-tls\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.747973 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.747896 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-sys\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.747973 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.747964 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-sys\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.747973 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.747963 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-wtmp\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.748395 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.748134 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1488becb-97bc-43e6-b85c-6c9c62dfe903-metrics-client-ca\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.748395 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.747809 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-root\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.748395 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.748246 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-accelerators-collector-config\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.750422 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.750396 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.750564 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.750545 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-tls\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.762892 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.762869 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlj2h\" (UniqueName: \"kubernetes.io/projected/1488becb-97bc-43e6-b85c-6c9c62dfe903-kube-api-access-wlj2h\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.766139 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.766053 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:51.926814 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.926777 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:15:51.938161 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:15:51.938127 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1488becb_97bc_43e6_b85c_6c9c62dfe903.slice/crio-562fe41876ab323191b043ffbc10fa295c9d3b5b3c3f24ff8919854c6be94834 WatchSource:0}: Error finding container 562fe41876ab323191b043ffbc10fa295c9d3b5b3c3f24ff8919854c6be94834: Status 404 returned error can't find the container with id 562fe41876ab323191b043ffbc10fa295c9d3b5b3c3f24ff8919854c6be94834 Nov 04 07:15:51.974357 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:51.974318 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:52.144987 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.144955 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:52.563654 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.563627 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:52.656203 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.656175 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 07:15:52.659641 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.659621 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.673876 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.673841 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Nov 04 07:15:52.675989 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.675966 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mgp2g\"" Nov 04 07:15:52.676100 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.676071 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Nov 04 07:15:52.676413 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.676284 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Nov 04 07:15:52.676413 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.676299 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Nov 04 07:15:52.676413 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.676349 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Nov 04 07:15:52.676413 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.676370 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Nov 04 07:15:52.676413 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.676383 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Nov 04 07:15:52.676702 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.676435 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Nov 04 07:15:52.676702 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.676446 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Nov 04 07:15:52.676882 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.676863 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 07:15:52.677831 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.677807 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:52.724448 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.724414 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ktbsj" event={"ID":"1488becb-97bc-43e6-b85c-6c9c62dfe903","Type":"ContainerStarted","Data":"562fe41876ab323191b043ffbc10fa295c9d3b5b3c3f24ff8919854c6be94834"} Nov 04 07:15:52.726384 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.726359 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8656d77c48-qkjzp" event={"ID":"1172079a-2931-4ecb-9196-68e6da14fd36","Type":"ContainerStarted","Data":"0a23c53d443309b3605893289bc1ca3cfff8d36518735ff04cef4ed499816242"} Nov 04 07:15:52.752094 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.752017 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-8656d77c48-qkjzp" podStartSLOduration=1.124725518 podStartE2EDuration="3.751994802s" totalImagesPullingTime="2.627269284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:15:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:52.751152561 +0000 UTC m=+59.840679541" watchObservedRunningTime="2026-11-04 07:15:52.751994802 +0000 UTC m=+59.841521745" Nov 04 07:15:52.756631 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.756598 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-config-volume\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.756771 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.756652 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-web-config\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.756832 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.756773 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.756832 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.756810 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.756832 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.756830 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqrrw\" (UniqueName: \"kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-kube-api-access-cqrrw\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.757131 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.756908 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-config-out\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.757131 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.756949 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.757131 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.757019 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.757131 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.757061 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.757316 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.757135 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.757316 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.757178 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.757316 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.757203 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.757316 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.757235 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.858339 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.858271 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.858339 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.858313 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.858550 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.858338 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqrrw\" (UniqueName: \"kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-kube-api-access-cqrrw\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.858550 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.858373 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-config-out\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.858550 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.858397 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.858550 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.858423 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.858747 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.858697 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.858791 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.858760 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.859003 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.858978 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.859076 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.859045 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.859134 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.859076 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.859134 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.859110 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.859242 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.859157 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-config-volume\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.859242 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.859212 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-web-config\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.859521 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.859477 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.862598 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.862572 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.862944 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.862924 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-config-out\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.863022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.862957 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.863284 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.863262 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.863401 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.863354 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.863672 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.863605 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.867064 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.863841 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-config-volume\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.867064 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.863955 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-web-config\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.867064 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.864036 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.867064 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.864266 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.874117 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.874097 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqrrw\" (UniqueName: \"kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-kube-api-access-cqrrw\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:52.902921 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.902896 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:52.990283 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:52.990259 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:15:53.093340 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:53.093313 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:53.227051 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:53.227021 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:53.307242 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:53.305110 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 07:15:53.708406 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:53.708377 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:53.731000 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:53.730968 2583 generic.go:356] "Generic (PLEG): container finished" podID="1488becb-97bc-43e6-b85c-6c9c62dfe903" containerID="a557e908a2b85aedd8fa6a97021d5833f02b97cb3d0c63e391b1179aec8c3ee3" exitCode=0 Nov 04 07:15:53.731131 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:53.731057 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ktbsj" event={"ID":"1488becb-97bc-43e6-b85c-6c9c62dfe903","Type":"ContainerDied","Data":"a557e908a2b85aedd8fa6a97021d5833f02b97cb3d0c63e391b1179aec8c3ee3"} Nov 04 07:15:53.732319 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:53.732296 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3632b027-6f63-4015-a775-b3a7d6d32247","Type":"ContainerStarted","Data":"2a0c856154e6e0535c5bc1b0fc56fd5269d08bac276190c963ee0aa3f660c6f4"} Nov 04 07:15:53.875353 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:53.875305 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:54.020066 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:54.019996 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:54.659595 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:54.659567 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:54.739401 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:54.739372 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ktbsj" event={"ID":"1488becb-97bc-43e6-b85c-6c9c62dfe903","Type":"ContainerStarted","Data":"9086c51f1b3ba41371380aa450daf560f5a9eb135388f21563a1e52b97c94e9d"} Nov 04 07:15:54.739571 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:54.739461 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ktbsj" event={"ID":"1488becb-97bc-43e6-b85c-6c9c62dfe903","Type":"ContainerStarted","Data":"b1cd9ad754b05d90f63d9bc99a792da14147dea10faeb6aefd321166c1e49399"} Nov 04 07:15:54.740858 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:54.740836 2583 generic.go:356] "Generic (PLEG): container finished" podID="3632b027-6f63-4015-a775-b3a7d6d32247" containerID="5782cea40175ed4b81900890fb9483ae7a3eccec289a6f13796e32865641f9ef" exitCode=0 Nov 04 07:15:54.740954 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:54.740882 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3632b027-6f63-4015-a775-b3a7d6d32247","Type":"ContainerDied","Data":"5782cea40175ed4b81900890fb9483ae7a3eccec289a6f13796e32865641f9ef"} Nov 04 07:15:54.766421 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:54.764864 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ktbsj" podStartSLOduration=2.722844148 podStartE2EDuration="3.764849334s" totalImagesPullingTime="967.195953ms" totalInitContainerRuntime="74.809233ms" isStatefulPod=true podCreationTimestamp="2026-11-04 07:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:54.764110192 +0000 UTC m=+61.853637144" watchObservedRunningTime="2026-11-04 07:15:54.764849334 +0000 UTC m=+61.854376273" Nov 04 07:15:54.783672 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:54.783642 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:54.899763 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:54.899740 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:55.100670 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:55.100593 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:56.310027 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:56.309987 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:56.415829 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:56.415798 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:56.516900 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:56.516841 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:56.618828 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:56.618796 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:56.751261 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:56.751221 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3632b027-6f63-4015-a775-b3a7d6d32247","Type":"ContainerStarted","Data":"025944af5d89221999b3afc40050fb1b38c43b5676d7d9e2882312b4e7ec55ee"} Nov 04 07:15:56.751447 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:56.751325 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3632b027-6f63-4015-a775-b3a7d6d32247","Type":"ContainerStarted","Data":"beee87a33b0b7ccbf013408ae24b9a234f241be68c6af640b7d3ed689d020802"} Nov 04 07:15:56.751447 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:56.751352 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3632b027-6f63-4015-a775-b3a7d6d32247","Type":"ContainerStarted","Data":"3ce87f1d9b05063aadb063e169720dd31a27f37360fd7e6aa7aa6c96a7bb5ce0"} Nov 04 07:15:56.751447 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:56.751398 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3632b027-6f63-4015-a775-b3a7d6d32247","Type":"ContainerStarted","Data":"0006278638bddf21a3ff6d49fc9d393a076eb64c9da8c81195994fc999b742d3"} Nov 04 07:15:56.751447 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:56.751422 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3632b027-6f63-4015-a775-b3a7d6d32247","Type":"ContainerStarted","Data":"eb9f92288969284551c442cc77eca55e870bed0bda8dac28eca9da37d2117fdf"} Nov 04 07:15:57.083864 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:57.083842 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:57.183054 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:57.183031 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:57.706993 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:57.706960 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qpt88" Nov 04 07:15:57.760750 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:57.760710 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3632b027-6f63-4015-a775-b3a7d6d32247","Type":"ContainerStarted","Data":"0ed20dd35961d617c1408d79b8677652ee6d9262c30f2092fdddd6ad301df166"} Nov 04 07:15:57.797124 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:57.795704 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.822595794 podStartE2EDuration="5.795685601s" totalImagesPullingTime="2.962336303s" totalInitContainerRuntime="10.753504ms" isStatefulPod=false podCreationTimestamp="2026-11-04 07:15:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:57.792950891 +0000 UTC m=+64.882477851" watchObservedRunningTime="2026-11-04 07:15:57.795685601 +0000 UTC m=+64.885212544" Nov 04 07:15:57.965788 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:57.965705 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:15:59.110361 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.110327 2583 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-d75fb8d4b-tj7rv"] Nov 04 07:15:59.111329 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.111301 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:59.113459 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.113441 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.114161 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.114141 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 07:15:59.122700 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.122677 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Nov 04 07:15:59.123971 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.123951 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:59.126056 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.126033 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d75fb8d4b-tj7rv"] Nov 04 07:15:59.212602 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.212570 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhnsv\" (UniqueName: \"kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv\") pod \"network-check-target-9qgxt\" (UID: \"97707149-0248-45a0-87d2-7347b960c2db\") " pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:59.212717 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.212607 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-trusted-ca-bundle\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.212717 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.212632 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-oauth-serving-cert\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.212717 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.212654 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzs4d\" (UniqueName: \"kubernetes.io/projected/30df6026-1baf-4fca-8c94-cb092397a676-kube-api-access-tzs4d\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.212846 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.212825 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-oauth-config\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.212909 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.212890 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-console-config\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.212951 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.212937 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs\") pod \"network-metrics-daemon-ldwvk\" (UID: \"17916760-be6c-47bf-8569-94ec0e36e304\") " pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:59.213017 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.212998 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-serving-cert\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.213067 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.213053 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-service-ca\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.213112 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.213076 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30df6026-1baf-4fca-8c94-cb092397a676-tmp\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.215925 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.215909 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 07:15:59.216011 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.215997 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 07:15:59.225284 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.225267 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs\") pod \"network-metrics-daemon-ldwvk\" (UID: \"17916760-be6c-47bf-8569-94ec0e36e304\") " pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:59.225666 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.225642 2583 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 07:15:59.235843 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.235824 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhnsv\" (UniqueName: \"kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv\") pod \"network-check-target-9qgxt\" (UID: \"97707149-0248-45a0-87d2-7347b960c2db\") " pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:59.314133 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.314112 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-service-ca\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.314209 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.314139 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30df6026-1baf-4fca-8c94-cb092397a676-tmp\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.314209 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.314169 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-trusted-ca-bundle\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.314322 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.314302 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-oauth-serving-cert\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.314364 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.314342 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzs4d\" (UniqueName: \"kubernetes.io/projected/30df6026-1baf-4fca-8c94-cb092397a676-kube-api-access-tzs4d\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.314413 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.314374 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-oauth-config\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.314458 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.314428 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-console-config\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.314533 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.314468 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-serving-cert\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.314639 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.314615 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30df6026-1baf-4fca-8c94-cb092397a676-tmp\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.314926 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.314903 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-service-ca\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.315022 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.315000 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-trusted-ca-bundle\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.315150 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.315130 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-oauth-serving-cert\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.315215 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.315170 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-console-config\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.316681 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.316663 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-oauth-config\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.316756 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.316728 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-serving-cert\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.326832 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.326810 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzs4d\" (UniqueName: \"kubernetes.io/projected/30df6026-1baf-4fca-8c94-cb092397a676-kube-api-access-tzs4d\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.349549 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.349529 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gnthh\"" Nov 04 07:15:59.356692 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.356672 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:15:59.358218 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.358201 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-zw52p\"" Nov 04 07:15:59.365465 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.365450 2583 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-hvg8n\"" Nov 04 07:15:59.366498 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.366481 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:15:59.373305 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.373288 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:15:59.425044 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.424812 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:15:59.596601 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.596572 2583 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:59.596754 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.596638 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:59.600926 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.600900 2583 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:59.641802 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.641776 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ldwvk"] Nov 04 07:15:59.644726 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:15:59.644699 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17916760_be6c_47bf_8569_94ec0e36e304.slice/crio-c3575da54d6411a09114faffa4f53a98f500552978ba887b705138e9eea75446 WatchSource:0}: Error finding container c3575da54d6411a09114faffa4f53a98f500552978ba887b705138e9eea75446: Status 404 returned error can't find the container with id c3575da54d6411a09114faffa4f53a98f500552978ba887b705138e9eea75446 Nov 04 07:15:59.651026 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.650964 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-m7gt8"] Nov 04 07:15:59.652371 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.652345 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9qgxt"] Nov 04 07:15:59.653880 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:15:59.653857 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97707149_0248_45a0_87d2_7347b960c2db.slice/crio-7690cf4f07ef529c2acc86e76277753fc1f838a5cbbdf28ff4176f47a487c129 WatchSource:0}: Error finding container 7690cf4f07ef529c2acc86e76277753fc1f838a5cbbdf28ff4176f47a487c129: Status 404 returned error can't find the container with id 7690cf4f07ef529c2acc86e76277753fc1f838a5cbbdf28ff4176f47a487c129 Nov 04 07:15:59.654881 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:15:59.654862 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa9b645b_69d5_47a7_8457_409364209023.slice/crio-4df9c74ae689e955ce0d7e956d9371ee0f78516b2d94770eb563f4266120b0ca WatchSource:0}: Error finding container 4df9c74ae689e955ce0d7e956d9371ee0f78516b2d94770eb563f4266120b0ca: Status 404 returned error can't find the container with id 4df9c74ae689e955ce0d7e956d9371ee0f78516b2d94770eb563f4266120b0ca Nov 04 07:15:59.711672 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.711649 2583 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d75fb8d4b-tj7rv"] Nov 04 07:15:59.713903 ip-10-0-131-6 kubenswrapper[2583]: W1104 07:15:59.713880 2583 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30df6026_1baf_4fca_8c94_cb092397a676.slice/crio-b158853477c105444b0009f84317f574595aad43b9046ecdb6eed47b76170255 WatchSource:0}: Error finding container b158853477c105444b0009f84317f574595aad43b9046ecdb6eed47b76170255: Status 404 returned error can't find the container with id b158853477c105444b0009f84317f574595aad43b9046ecdb6eed47b76170255 Nov 04 07:15:59.769021 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.768998 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9qgxt" event={"ID":"97707149-0248-45a0-87d2-7347b960c2db","Type":"ContainerStarted","Data":"7690cf4f07ef529c2acc86e76277753fc1f838a5cbbdf28ff4176f47a487c129"} Nov 04 07:15:59.770118 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.770091 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-m7gt8" event={"ID":"fa9b645b-69d5-47a7-8457-409364209023","Type":"ContainerStarted","Data":"4df9c74ae689e955ce0d7e956d9371ee0f78516b2d94770eb563f4266120b0ca"} Nov 04 07:15:59.771633 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.771612 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d75fb8d4b-tj7rv" event={"ID":"30df6026-1baf-4fca-8c94-cb092397a676","Type":"ContainerStarted","Data":"eebe19ee00c2d25a80cd76c9c1da0715ef261bf00407d399ca0b8a1560f74648"} Nov 04 07:15:59.771720 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.771648 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d75fb8d4b-tj7rv" event={"ID":"30df6026-1baf-4fca-8c94-cb092397a676","Type":"ContainerStarted","Data":"b158853477c105444b0009f84317f574595aad43b9046ecdb6eed47b76170255"} Nov 04 07:15:59.772929 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.772911 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ldwvk" event={"ID":"17916760-be6c-47bf-8569-94ec0e36e304","Type":"ContainerStarted","Data":"c3575da54d6411a09114faffa4f53a98f500552978ba887b705138e9eea75446"} Nov 04 07:15:59.777240 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.777219 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:15:59.792992 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:15:59.792937 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-d75fb8d4b-tj7rv" podStartSLOduration=0.792922179 podStartE2EDuration="792.922179ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:15:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:15:59.792383752 +0000 UTC m=+66.881910711" watchObservedRunningTime="2026-11-04 07:15:59.792922179 +0000 UTC m=+66.882449117" Nov 04 07:16:00.929180 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:00.929153 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:01.047233 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:01.047142 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:01.782608 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:01.782564 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ldwvk" event={"ID":"17916760-be6c-47bf-8569-94ec0e36e304","Type":"ContainerStarted","Data":"01b97700344bab97a1caee07515d644afa80eda5cf87a9fe2608d74cc3b14770"} Nov 04 07:16:01.782783 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:01.782636 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ldwvk" event={"ID":"17916760-be6c-47bf-8569-94ec0e36e304","Type":"ContainerStarted","Data":"bc19533c3f92df8fb461d71cfd2e899d9ca6a658a4f6f4634712bd250ca67d9f"} Nov 04 07:16:01.805356 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:01.805308 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ldwvk" podStartSLOduration=66.704263057 podStartE2EDuration="1m7.805294896s" totalImagesPullingTime="1.101031839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:14:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:16:01.804387303 +0000 UTC m=+68.893914242" watchObservedRunningTime="2026-11-04 07:16:01.805294896 +0000 UTC m=+68.894821834" Nov 04 07:16:01.948850 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:01.948818 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:02.080751 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:02.080665 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:02.197475 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:02.197438 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:02.314826 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:02.314792 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:02.473804 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:02.473764 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:02.744582 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:02.741401 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:02.863800 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:02.863759 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:03.060769 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:03.060722 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:03.263995 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:03.263957 2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:03.792490 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:03.792455 2583 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9qgxt" event={"ID":"97707149-0248-45a0-87d2-7347b960c2db","Type":"ContainerStarted","Data":"6c929eb5ead467ca0c87ff0a517017e9e4ed22dada4254ec10fede7be48e738a"} Nov 04 07:16:03.792676 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:03.792600 2583 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:16:03.814471 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:03.814421 2583 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9qgxt" podStartSLOduration=66.450302521 podStartE2EDuration="1m9.814381673s" totalImagesPullingTime="3.364079152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:14:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:16:03.813295972 +0000 UTC m=+70.902822915" watchObservedRunningTime="2026-11-04 07:16:03.814381673 +0000 UTC m=+70.903908611" Nov 04 07:16:04.599909 ip-10-0-131-6 kubenswrapper[2583]: I1104 07:16:04.599800 2583 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 07:16:04.599951 ip-10-0-131-6 systemd[1]: Stopping Kubernetes Kubelet... Nov 04 07:16:04.611859 ip-10-0-131-6 systemd[1]: kubelet.service: Deactivated successfully. Nov 04 07:16:04.612029 ip-10-0-131-6 systemd[1]: Stopped Kubernetes Kubelet. Nov 04 07:16:04.612080 ip-10-0-131-6 systemd[1]: kubelet.service: Consumed 3.886s CPU time. Nov 04 07:16:04.623178 ip-10-0-131-6 systemd[1]: Starting Kubernetes Kubelet... Nov 04 07:16:04.704661 ip-10-0-131-6 kubenswrapper[5385]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 07:16:04.704661 ip-10-0-131-6 kubenswrapper[5385]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 07:16:04.704661 ip-10-0-131-6 kubenswrapper[5385]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 07:16:04.704661 ip-10-0-131-6 kubenswrapper[5385]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 07:16:04.707468 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707346 5385 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 07:16:04.707468 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707455 5385 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 07:16:04.707468 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707460 5385 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 07:16:04.707468 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707465 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 07:16:04.707468 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707470 5385 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 07:16:04.707468 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707475 5385 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 07:16:04.707468 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707479 5385 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707483 5385 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707488 5385 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707492 5385 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707497 5385 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707517 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707522 5385 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707527 5385 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707531 5385 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707535 5385 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707541 5385 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707546 5385 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707550 5385 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707556 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707560 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707565 5385 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707569 5385 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707574 5385 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707579 5385 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707583 5385 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 07:16:04.707825 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707587 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707589 5385 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707597 5385 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707600 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707603 5385 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707606 5385 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707609 5385 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707612 5385 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707616 5385 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707621 5385 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707625 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707628 5385 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707632 5385 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707634 5385 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707637 5385 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707640 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707644 5385 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707648 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707653 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 07:16:04.708307 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707657 5385 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707661 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707665 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707669 5385 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707673 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707686 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707690 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707694 5385 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707699 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707705 5385 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707709 5385 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707714 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707718 5385 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707722 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707727 5385 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707731 5385 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707736 5385 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707741 5385 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707745 5385 feature_gate.go:386] unrecognized feature gate: Example Nov 04 07:16:04.708913 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707749 5385 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707755 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707760 5385 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707764 5385 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707768 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707772 5385 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707777 5385 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707781 5385 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707786 5385 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707791 5385 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707796 5385 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707801 5385 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707805 5385 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707811 5385 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707816 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707820 5385 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707824 5385 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707831 5385 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707838 5385 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 07:16:04.709433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707844 5385 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707849 5385 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707854 5385 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707859 5385 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707863 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707868 5385 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707874 5385 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707879 5385 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707884 5385 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707888 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707892 5385 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707898 5385 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707903 5385 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707907 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707912 5385 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707917 5385 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707922 5385 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707926 5385 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707930 5385 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707934 5385 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 07:16:04.709912 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707939 5385 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707944 5385 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.707948 5385 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709240 5385 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709251 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709256 5385 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709261 5385 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709266 5385 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709270 5385 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709275 5385 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709279 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709283 5385 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709288 5385 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709293 5385 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709297 5385 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709302 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709306 5385 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709311 5385 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709315 5385 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709319 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 07:16:04.710380 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709324 5385 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709328 5385 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709332 5385 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709337 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709343 5385 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709347 5385 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709352 5385 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709357 5385 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709374 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709378 5385 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709383 5385 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709387 5385 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709395 5385 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709400 5385 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709406 5385 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709411 5385 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709416 5385 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709421 5385 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709425 5385 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 07:16:04.710863 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709430 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709435 5385 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709440 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709445 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709450 5385 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709454 5385 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709458 5385 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709463 5385 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709467 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709472 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709476 5385 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709481 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709485 5385 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709489 5385 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709495 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709520 5385 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709525 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709530 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709536 5385 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709541 5385 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 07:16:04.711337 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709546 5385 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709550 5385 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709554 5385 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709559 5385 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709563 5385 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709568 5385 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709573 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709578 5385 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709582 5385 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709587 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709591 5385 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709596 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709600 5385 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709604 5385 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709609 5385 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709613 5385 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709618 5385 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709622 5385 feature_gate.go:386] unrecognized feature gate: Example Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709626 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709630 5385 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 07:16:04.711831 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709635 5385 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709639 5385 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709644 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709648 5385 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709653 5385 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709657 5385 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709662 5385 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709668 5385 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709675 5385 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709682 5385 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709688 5385 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709693 5385 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709700 5385 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709704 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709709 5385 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709713 5385 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709718 5385 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709722 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709729 5385 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 07:16:04.712310 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709734 5385 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709738 5385 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709743 5385 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709747 5385 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709751 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709756 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709760 5385 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709764 5385 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709768 5385 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709772 5385 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.709776 5385 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710636 5385 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710649 5385 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710655 5385 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710660 5385 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710664 5385 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710669 5385 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710674 5385 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710678 5385 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710683 5385 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 07:16:04.712900 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710688 5385 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710693 5385 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710697 5385 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710702 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710707 5385 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710711 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710715 5385 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710720 5385 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710724 5385 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710728 5385 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710732 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710736 5385 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710741 5385 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710745 5385 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710749 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710753 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710758 5385 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710762 5385 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710767 5385 feature_gate.go:386] unrecognized feature gate: Example Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710772 5385 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 07:16:04.713382 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710776 5385 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710780 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710785 5385 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710789 5385 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710793 5385 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710798 5385 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710802 5385 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710807 5385 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710811 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710816 5385 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710821 5385 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710825 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710830 5385 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710834 5385 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710839 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710844 5385 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710848 5385 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710853 5385 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710857 5385 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710863 5385 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 07:16:04.713910 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710867 5385 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710873 5385 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710877 5385 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710883 5385 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710887 5385 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710891 5385 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710895 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710900 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710904 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710908 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710913 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710917 5385 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710921 5385 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710925 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710930 5385 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710935 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710940 5385 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710945 5385 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710949 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 07:16:04.714427 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710954 5385 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710958 5385 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710962 5385 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710966 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710971 5385 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710975 5385 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710979 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710986 5385 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710992 5385 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.710996 5385 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711000 5385 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711006 5385 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711010 5385 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711015 5385 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711019 5385 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711025 5385 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711031 5385 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711037 5385 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711043 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 07:16:04.715221 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711049 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711054 5385 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711058 5385 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711063 5385 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711067 5385 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711072 5385 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711076 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711080 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711085 5385 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711089 5385 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711093 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711097 5385 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711101 5385 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711106 5385 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711110 5385 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711115 5385 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711119 5385 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711123 5385 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.711128 5385 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711269 5385 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 07:16:04.715989 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711284 5385 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711293 5385 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711300 5385 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711309 5385 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711315 5385 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711323 5385 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711330 5385 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711335 5385 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711340 5385 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711348 5385 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711354 5385 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711359 5385 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711364 5385 flags.go:64] FLAG: --cgroup-root="" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711369 5385 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711374 5385 flags.go:64] FLAG: --client-ca-file="" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711378 5385 flags.go:64] FLAG: --cloud-provider="external" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711383 5385 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711389 5385 flags.go:64] FLAG: --cluster-domain="" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711394 5385 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711399 5385 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711404 5385 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711410 5385 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711416 5385 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 07:16:04.716719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711421 5385 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711427 5385 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711432 5385 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711437 5385 flags.go:64] FLAG: --contention-profiling="false" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711442 5385 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711447 5385 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711452 5385 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711457 5385 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711464 5385 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711469 5385 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711474 5385 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711479 5385 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711486 5385 flags.go:64] FLAG: --enable-server="true" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711491 5385 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711497 5385 flags.go:64] FLAG: --event-burst="100" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711520 5385 flags.go:64] FLAG: --event-qps="50" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711525 5385 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711530 5385 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711539 5385 flags.go:64] FLAG: --eviction-hard="" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711546 5385 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711551 5385 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711556 5385 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711561 5385 flags.go:64] FLAG: --eviction-soft="" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711567 5385 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711571 5385 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 07:16:04.717338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711576 5385 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711581 5385 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711587 5385 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711592 5385 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711597 5385 flags.go:64] FLAG: --feature-gates="" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711603 5385 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711608 5385 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711613 5385 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711619 5385 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711624 5385 flags.go:64] FLAG: --healthz-port="10248" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711629 5385 flags.go:64] FLAG: --help="false" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711634 5385 flags.go:64] FLAG: --hostname-override="ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711639 5385 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711644 5385 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711650 5385 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711656 5385 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711662 5385 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711667 5385 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711671 5385 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711678 5385 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711682 5385 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711687 5385 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711693 5385 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711699 5385 flags.go:64] FLAG: --kube-reserved="" Nov 04 07:16:04.717970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711704 5385 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711711 5385 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711717 5385 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711721 5385 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711726 5385 flags.go:64] FLAG: --lock-file="" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711731 5385 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711736 5385 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711742 5385 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711751 5385 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711756 5385 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711761 5385 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711766 5385 flags.go:64] FLAG: --logging-format="text" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711771 5385 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711777 5385 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711782 5385 flags.go:64] FLAG: --manifest-url="" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711786 5385 flags.go:64] FLAG: --manifest-url-header="" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711794 5385 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711799 5385 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711805 5385 flags.go:64] FLAG: --max-pods="110" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711810 5385 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711816 5385 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711821 5385 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711826 5385 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711831 5385 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711836 5385 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 07:16:04.719848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711841 5385 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711853 5385 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711860 5385 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711865 5385 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711870 5385 flags.go:64] FLAG: --pod-cidr="" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711875 5385 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711880 5385 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711886 5385 flags.go:64] FLAG: --pods-per-core="0" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711893 5385 flags.go:64] FLAG: --port="10250" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711898 5385 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711903 5385 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-00844c6faf71f481b" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711908 5385 flags.go:64] FLAG: --qos-reserved="" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711914 5385 flags.go:64] FLAG: --read-only-port="10255" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711919 5385 flags.go:64] FLAG: --register-node="true" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711924 5385 flags.go:64] FLAG: --register-with-taints="" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711931 5385 flags.go:64] FLAG: --registry-burst="10" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711936 5385 flags.go:64] FLAG: --registry-qps="5" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711941 5385 flags.go:64] FLAG: --reserved-cpus="" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711946 5385 flags.go:64] FLAG: --reserved-memory="" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711952 5385 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711957 5385 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711962 5385 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711968 5385 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711973 5385 flags.go:64] FLAG: --runonce="false" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711978 5385 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711984 5385 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711989 5385 flags.go:64] FLAG: --seccomp-default="false" Nov 04 07:16:04.720964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711994 5385 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.711999 5385 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712004 5385 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712009 5385 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712015 5385 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712020 5385 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712025 5385 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712032 5385 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712037 5385 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712042 5385 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712047 5385 flags.go:64] FLAG: --system-cgroups="" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712052 5385 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712062 5385 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712069 5385 flags.go:64] FLAG: --tls-cert-file="" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712075 5385 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712081 5385 flags.go:64] FLAG: --tls-min-version="" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712086 5385 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712091 5385 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712096 5385 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712102 5385 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712107 5385 flags.go:64] FLAG: --v="2" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712114 5385 flags.go:64] FLAG: --version="false" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712121 5385 flags.go:64] FLAG: --vmodule="" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712127 5385 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712133 5385 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 07:16:04.722030 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712356 5385 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712364 5385 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712369 5385 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712375 5385 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712380 5385 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712384 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712390 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712395 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712399 5385 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712404 5385 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712408 5385 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712413 5385 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712418 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712422 5385 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712428 5385 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712433 5385 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712438 5385 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712442 5385 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712447 5385 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712451 5385 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 07:16:04.722990 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712458 5385 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712464 5385 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712469 5385 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712473 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712478 5385 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712482 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712487 5385 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712492 5385 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712496 5385 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712519 5385 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712524 5385 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712528 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712533 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712537 5385 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712542 5385 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712547 5385 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712552 5385 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712556 5385 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712560 5385 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712565 5385 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 07:16:04.723838 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712569 5385 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712573 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712578 5385 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712582 5385 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712587 5385 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712592 5385 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712596 5385 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712601 5385 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712606 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712610 5385 feature_gate.go:386] unrecognized feature gate: Example Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712614 5385 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712619 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712626 5385 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712630 5385 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712636 5385 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712640 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712645 5385 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712649 5385 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712654 5385 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712659 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712664 5385 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 07:16:04.724696 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712668 5385 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712672 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712677 5385 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712681 5385 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712686 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712690 5385 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712695 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712699 5385 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712703 5385 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712707 5385 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712712 5385 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712716 5385 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712721 5385 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712725 5385 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712730 5385 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712734 5385 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712739 5385 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712743 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712747 5385 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 07:16:04.725592 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712752 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712757 5385 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712761 5385 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712766 5385 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712772 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712777 5385 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712781 5385 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712786 5385 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712791 5385 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712795 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712800 5385 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712804 5385 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712808 5385 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712813 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712818 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712822 5385 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712826 5385 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712831 5385 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712835 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 07:16:04.726438 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712840 5385 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 07:16:04.727276 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712846 5385 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 07:16:04.727276 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712853 5385 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 07:16:04.727276 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712861 5385 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 07:16:04.727276 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712866 5385 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 07:16:04.727276 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712872 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 07:16:04.727276 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.712877 5385 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 07:16:04.727276 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.712926 5385 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 07:16:04.727276 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713004 5385 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 07:16:04.727276 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713012 5385 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 07:16:04.727276 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713019 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 07:16:04.727276 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713024 5385 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 07:16:04.727276 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713029 5385 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 07:16:04.727276 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713033 5385 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 07:16:04.727276 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713037 5385 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 07:16:04.727276 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713042 5385 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713047 5385 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713052 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713057 5385 feature_gate.go:386] unrecognized feature gate: Example Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713061 5385 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713066 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713071 5385 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713076 5385 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713080 5385 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713084 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713088 5385 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713093 5385 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713098 5385 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713102 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713107 5385 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713111 5385 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713115 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713120 5385 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713124 5385 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713129 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 07:16:04.727925 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713133 5385 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713138 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713145 5385 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713149 5385 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713154 5385 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713158 5385 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713163 5385 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713167 5385 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713171 5385 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713176 5385 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713180 5385 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713185 5385 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713189 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713194 5385 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713199 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713204 5385 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713208 5385 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713213 5385 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713218 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 07:16:04.728722 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713223 5385 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713227 5385 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713232 5385 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713236 5385 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713240 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713245 5385 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713250 5385 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713254 5385 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713258 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713263 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713267 5385 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713272 5385 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713276 5385 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713281 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713285 5385 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713294 5385 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713301 5385 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713307 5385 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 07:16:04.729445 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713312 5385 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713317 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713322 5385 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713327 5385 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713332 5385 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713336 5385 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713340 5385 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713345 5385 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713349 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713355 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713360 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713364 5385 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713369 5385 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713373 5385 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713378 5385 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713383 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713388 5385 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713393 5385 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713398 5385 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713402 5385 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 07:16:04.730093 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713406 5385 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713411 5385 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713415 5385 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713420 5385 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713425 5385 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713429 5385 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713434 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713438 5385 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713443 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713449 5385 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713454 5385 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713458 5385 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713462 5385 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713467 5385 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713472 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713476 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713481 5385 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713485 5385 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713490 5385 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 07:16:04.730757 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713497 5385 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713519 5385 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.713524 5385 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.719526 5385 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.719545 5385 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.719569 5385 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.719577 5385 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719663 5385 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719671 5385 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719678 5385 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719684 5385 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719688 5385 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719693 5385 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719697 5385 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719702 5385 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719706 5385 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719711 5385 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719715 5385 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719720 5385 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719724 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 07:16:04.731311 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719732 5385 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719736 5385 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719741 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719745 5385 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719751 5385 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719755 5385 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719761 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719765 5385 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719770 5385 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719774 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719779 5385 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719783 5385 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719788 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719792 5385 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719797 5385 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719802 5385 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719806 5385 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719811 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719815 5385 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719820 5385 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 07:16:04.731951 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719824 5385 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719829 5385 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719835 5385 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719839 5385 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719844 5385 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719848 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719852 5385 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719856 5385 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719860 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719864 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719869 5385 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719873 5385 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719877 5385 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719883 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719888 5385 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719892 5385 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719897 5385 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719902 5385 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719907 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 07:16:04.732433 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719912 5385 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719917 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719921 5385 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719925 5385 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719931 5385 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719935 5385 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719939 5385 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719944 5385 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719948 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719953 5385 feature_gate.go:386] unrecognized feature gate: Example Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719958 5385 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719962 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719967 5385 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719972 5385 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719977 5385 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719981 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719985 5385 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719990 5385 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719994 5385 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 07:16:04.732905 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.719999 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720003 5385 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720008 5385 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720012 5385 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720017 5385 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720021 5385 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720026 5385 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720030 5385 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720035 5385 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720040 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720044 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720049 5385 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720055 5385 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720059 5385 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720064 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720068 5385 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720073 5385 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720077 5385 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720081 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720086 5385 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 07:16:04.733392 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720090 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 07:16:04.733938 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720095 5385 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 07:16:04.733938 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720099 5385 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 07:16:04.733938 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720106 5385 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 07:16:04.733938 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720114 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 07:16:04.733938 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720120 5385 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 07:16:04.733938 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720124 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 07:16:04.733938 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720129 5385 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 07:16:04.733938 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720133 5385 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 07:16:04.733938 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720137 5385 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 07:16:04.733938 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720142 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 07:16:04.733938 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720146 5385 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 07:16:04.733938 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720151 5385 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 07:16:04.733938 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720158 5385 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 07:16:04.733938 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720164 5385 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 07:16:04.733938 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.720206 5385 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720431 5385 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720441 5385 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720446 5385 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720450 5385 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720455 5385 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720461 5385 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720465 5385 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720470 5385 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720475 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720480 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720484 5385 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720488 5385 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720493 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720497 5385 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720521 5385 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720526 5385 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720531 5385 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720535 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720540 5385 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 07:16:04.734329 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720544 5385 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720549 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720553 5385 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720557 5385 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720562 5385 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720566 5385 feature_gate.go:386] unrecognized feature gate: Example Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720570 5385 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720575 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720579 5385 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720583 5385 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720587 5385 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720592 5385 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720596 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720601 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720605 5385 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720609 5385 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720614 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720618 5385 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720623 5385 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 07:16:04.734845 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720629 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720633 5385 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720638 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720642 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720647 5385 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720654 5385 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720661 5385 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720666 5385 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720671 5385 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720675 5385 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720680 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720684 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720689 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720693 5385 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720697 5385 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720701 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720706 5385 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720710 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720715 5385 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 07:16:04.735308 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720719 5385 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720724 5385 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720729 5385 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720733 5385 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720738 5385 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720742 5385 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720746 5385 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720756 5385 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720760 5385 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720766 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720770 5385 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720775 5385 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720779 5385 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720784 5385 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720789 5385 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720793 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720797 5385 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720802 5385 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720806 5385 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720810 5385 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 07:16:04.736066 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720815 5385 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720819 5385 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720824 5385 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720828 5385 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720835 5385 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720841 5385 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720847 5385 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720851 5385 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720856 5385 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720861 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720865 5385 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720870 5385 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720875 5385 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720879 5385 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720883 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720889 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720893 5385 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720898 5385 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720903 5385 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 07:16:04.736843 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720909 5385 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 07:16:04.737414 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720913 5385 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 07:16:04.737414 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720918 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 07:16:04.737414 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720922 5385 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 07:16:04.737414 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720927 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 07:16:04.737414 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720931 5385 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 07:16:04.737414 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720936 5385 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 07:16:04.737414 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720940 5385 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 07:16:04.737414 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720946 5385 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 07:16:04.737414 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.720950 5385 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 07:16:04.737414 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.720990 5385 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 07:16:04.737414 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721043 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 07:16:04.737414 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721051 5385 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 07:16:04.737414 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721056 5385 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 07:16:04.737414 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721063 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 07:16:04.737414 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721068 5385 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 07:16:04.737414 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721072 5385 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721077 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721081 5385 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721086 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721090 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721095 5385 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721102 5385 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721107 5385 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721112 5385 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721117 5385 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721121 5385 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721126 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721131 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721135 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721139 5385 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721144 5385 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721148 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721155 5385 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721159 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 07:16:04.737983 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721164 5385 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721169 5385 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721173 5385 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721177 5385 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721182 5385 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721186 5385 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721192 5385 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721197 5385 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721201 5385 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721206 5385 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721211 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721215 5385 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721220 5385 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721224 5385 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721228 5385 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721235 5385 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721239 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721244 5385 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721248 5385 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721253 5385 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 07:16:04.738463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721257 5385 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721262 5385 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721266 5385 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721271 5385 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721275 5385 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721282 5385 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721288 5385 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721294 5385 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721299 5385 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721304 5385 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.721310 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.722975 5385 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.722986 5385 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.722991 5385 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.722996 5385 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723001 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723008 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723012 5385 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723017 5385 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 07:16:04.739012 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723024 5385 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723029 5385 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723034 5385 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723038 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723043 5385 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723047 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723052 5385 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723056 5385 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723061 5385 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723066 5385 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723071 5385 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723076 5385 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723081 5385 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723086 5385 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723090 5385 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723095 5385 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723102 5385 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723107 5385 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723111 5385 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723116 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 07:16:04.739570 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723120 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723125 5385 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723129 5385 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723137 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723141 5385 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723146 5385 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723150 5385 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723155 5385 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723159 5385 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723164 5385 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723168 5385 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723173 5385 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723177 5385 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723183 5385 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723187 5385 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723202 5385 feature_gate.go:386] unrecognized feature gate: Example Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723207 5385 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723212 5385 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723217 5385 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723221 5385 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 07:16:04.740055 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723226 5385 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 07:16:04.740547 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723230 5385 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 07:16:04.740547 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:04.723235 5385 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 07:16:04.740547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.723338 5385 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 07:16:04.740547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.724974 5385 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Nov 04 07:16:04.740547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.725047 5385 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Nov 04 07:16:04.740547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.725298 5385 server.go:1014] "Starting client certificate rotation" Nov 04 07:16:04.740547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.725439 5385 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 07:16:04.740547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.725494 5385 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 07:09:54 +0000 UTC" deadline="2028-06-10 04:04:49.198799486 +0000 UTC" Nov 04 07:16:04.740547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.725540 5385 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14012h48m44.47326197s" Nov 04 07:16:04.740547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.726047 5385 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 07:16:04.740547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.727735 5385 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 07:16:04.740547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.731157 5385 log.go:25] "Validated CRI v1 runtime API" Nov 04 07:16:04.740547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.733029 5385 log.go:25] "Validated CRI v1 image API" Nov 04 07:16:04.740547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.733823 5385 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 07:16:04.740547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.737706 5385 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ae07571e-34ea-4bf3-b8c3-f6d9767bd153:/dev/nvme0n1p4 c4f37c9e-820d-483e-b1ba-109835749dd0:/dev/nvme0n1p3] Nov 04 07:16:04.740973 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.737734 5385 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0920f6d09f5542e9ff46a73d1e32487fa63e11fe8640270165d7c37419792f2b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0920f6d09f5542e9ff46a73d1e32487fa63e11fe8640270165d7c37419792f2b/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/192706c9c89870c90332435973b8d137b0548ea06c810e67817775533b83ccc9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/192706c9c89870c90332435973b8d137b0548ea06c810e67817775533b83ccc9/userdata/shm major:0 minor:97 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2a0c856154e6e0535c5bc1b0fc56fd5269d08bac276190c963ee0aa3f660c6f4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2a0c856154e6e0535c5bc1b0fc56fd5269d08bac276190c963ee0aa3f660c6f4/userdata/shm major:0 minor:300 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/42519616ec898f3cc24635c600c6319af9749812f96791d88ba1f4dbd5823264/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/42519616ec898f3cc24635c600c6319af9749812f96791d88ba1f4dbd5823264/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4c6a17b097876c7082ce87b31d331614041295afbbb1e5d1d28f674afc0e94b7/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4c6a17b097876c7082ce87b31d331614041295afbbb1e5d1d28f674afc0e94b7/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4df9c74ae689e955ce0d7e956d9371ee0f78516b2d94770eb563f4266120b0ca/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4df9c74ae689e955ce0d7e956d9371ee0f78516b2d94770eb563f4266120b0ca/userdata/shm major:0 minor:351 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4f9d24ff5fa1251fad621af3e696722792385fda408c20e827e43db5e133ec3b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4f9d24ff5fa1251fad621af3e696722792385fda408c20e827e43db5e133ec3b/userdata/shm major:0 minor:239 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/562fe41876ab323191b043ffbc10fa295c9d3b5b3c3f24ff8919854c6be94834/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/562fe41876ab323191b043ffbc10fa295c9d3b5b3c3f24ff8919854c6be94834/userdata/shm major:0 minor:278 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5b287a145f727b1a4a1a27b9710ee3ff2ed4cf8a7797b20a959e36933e317fc8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5b287a145f727b1a4a1a27b9710ee3ff2ed4cf8a7797b20a959e36933e317fc8/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5fc708252796083f69ed7f8766d4018af524ce264be29bba95ae7dac9aa83d4e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5fc708252796083f69ed7f8766d4018af524ce264be29bba95ae7dac9aa83d4e/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/684c4196729a5a5c95e2abb153f5df3b2a2c0f984ecc4c757a346a9f663b5e7a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/684c4196729a5a5c95e2abb153f5df3b2a2c0f984ecc4c757a346a9f663b5e7a/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/709bfc975391d29847dce94faba2126dbd67b2d87a85fc353b0f0245dd41f34e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/709bfc975391d29847dce94faba2126dbd67b2d87a85fc353b0f0245dd41f34e/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/74dde8637d7b8fd440905c117fd8e135e7a24ffb9b04c353c7920db3f179b5b2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/74dde8637d7b8fd440905c117fd8e135e7a24ffb9b04c353c7920db3f179b5b2/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7690cf4f07ef529c2acc86e76277753fc1f838a5cbbdf28ff4176f47a487c129/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7690cf4f07ef529c2acc86e76277753fc1f838a5cbbdf28ff4176f47a487c129/userdata/shm major:0 minor:353 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a2a9454ff0a9aaee20e782323e0e1f9f2db0f10bed694f3003567300d327e1ec/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a2a9454ff0a9aaee20e782323e0e1f9f2db0f10bed694f3003567300d327e1ec/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b158853477c105444b0009f84317f574595aad43b9046ecdb6eed47b76170255/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b158853477c105444b0009f84317f574595aad43b9046ecdb6eed47b76170255/userdata/shm major:0 minor:355 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b863cc9011c198aadf7cc1bf200574267b9da8b94ddfc802cd35e134a423fd52/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b863cc9011c198aadf7cc1bf200574267b9da8b94ddfc802cd35e134a423fd52/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c3575da54d6411a09114faffa4f53a98f500552978ba887b705138e9eea75446/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c3575da54d6411a09114faffa4f53a98f500552978ba887b705138e9eea75446/userdata/shm major:0 minor:349 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e1efd0dbd7ffad01b836b1df99461789e63e533d166951a6420a7a5596cb62bf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e1efd0dbd7ffad01b836b1df99461789e63e533d166951a6420a7a5596cb62bf/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e831a99c631e426e454e7baf13c0458c225215cf935034bd771343ed086fc2c4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e831a99c631e426e454e7baf13c0458c225215cf935034bd771343ed086fc2c4/userdata/shm major:0 minor:83 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fc16b866b67f1e6c70bb4ee91fdc2a6fb1b7120131aaaee82fb4f1b3a30bcc0d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fc16b866b67f1e6c70bb4ee91fdc2a6fb1b7120131aaaee82fb4f1b3a30bcc0d/userdata/shm major:0 minor:271 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1172079a-2931-4ecb-9196-68e6da14fd36/volumes/kubernetes.io~projected/kube-api-access-hskwv:{mountpoint:/var/lib/kubelet/pods/1172079a-2931-4ecb-9196-68e6da14fd36/volumes/kubernetes.io~projected/kube-api-access-hskwv major:0 minor:270 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1172079a-2931-4ecb-9196-68e6da14fd36/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/1172079a-2931-4ecb-9196-68e6da14fd36/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:265 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1172079a-2931-4ecb-9196-68e6da14fd36/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/1172079a-2931-4ecb-9196-68e6da14fd36/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:269 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1488becb-97bc-43e6-b85c-6c9c62dfe903/volumes/kubernetes.io~projected/kube-api-access-wlj2h:{mountpoint:/var/lib/kubelet/pods/1488becb-97bc-43e6-b85c-6c9c62dfe903/volumes/kubernetes.io~projected/kube-api-access-wlj2h major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1488becb-97bc-43e6-b85c-6c9c62dfe903/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/1488becb-97bc-43e6-b85c-6c9c62dfe903/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:275 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1488becb-97bc-43e6-b85c-6c9c62dfe903/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/1488becb-97bc-43e6-b85c-6c9c62dfe903/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:276 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/17916760-be6c-47bf-8569-94ec0e36e304/volumes/kubernetes.io~projected/kube-api-access-lsljv:{mountpoint:/var/lib/kubelet/pods/17916760-be6c-47bf-8569-94ec0e36e304/volumes/kubernetes.io~projected/kube-api-access-lsljv major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/17916760-be6c-47bf-8569-94ec0e36e304/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/17916760-be6c-47bf-8569-94ec0e36e304/volumes/kubernetes.io~secret/metrics-certs major:0 minor:344 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/19b2ab56-a868-4570-b098-37c517da236a/volumes/kubernetes.io~projected/kube-api-access-f2dxb:{mountpoint:/var/lib/kubelet/pods/19b2ab56-a868-4570-b098-37c517da236a/volumes/kubernetes.io~projected/kube-api-access-f2dxb major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30df6026-1baf-4fca-8c94-cb092397a676/volumes/kubernetes.io~projected/kube-api-access-tzs4d:{mountpoint:/var/lib/kubelet/pods/30df6026-1baf-4fca-8c94-cb092397a676/volumes/kubernetes.io~projected/kube-api-access-tzs4d major:0 minor:348 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30df6026-1baf-4fca-8c94-cb092397a676/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/30df6026-1baf-4fca-8c94-cb092397a676/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:346 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30df6026-1baf-4fca-8c94-cb092397a676/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/30df6026-1baf-4fca-8c94-cb092397a676/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:347 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad/volumes/kubernetes.io~projected/kube-api-access-rtzpm:{mountpoint:/var/lib/kubelet/pods/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad/volumes/kubernetes.io~projected/kube-api-access-rtzpm major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:291 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volume-subpaths/web-config/alertmanager/9 major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volume-subpaths/web-config/config-reloader/6 major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~empty-dir/config-out major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~projected/kube-api-access-cqrrw:{mountpoint:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~projected/kube-api-access-cqrrw major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~projected/tls-assets major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:291 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/config-volume major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:292 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:284 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/web-config major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/508fee92-08e3-49ab-8b16-3add538b11eb/volumes/kubernetes.io~projected/kube-api-access-j4rg8:{mountpoint:/var/lib/kubelet/pods/508fee92-08e3-49ab-8b16-3add538b11eb/volumes/kubernetes.io~projected/kube-api-access-j4rg8 major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/508fee92-08e3-49ab-8b16-3add538b11eb/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/508fee92-08e3-49ab-8b16-3add538b11eb/volumes/kubernetes.io~secret/metrics-tls major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51538327-34ef-409b-b76a-6963b7254eec/volumes/kubernetes.io~projected/kube-api-access-7m7f6:{mountpoint:/var/lib/kubelet/pods/51538327-34ef-409b-b76a-6963b7254eec/volumes/kubernetes.io~projected/kube-api-access-7m7f6 major:0 minor:238 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51538327-34ef-409b-b76a-6963b7254eec/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/51538327-34ef-409b-b76a-6963b7254eec/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:237 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51538327-34ef-409b-b76a-6963b7254eec/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/51538327-34ef-409b-b76a-6963b7254eec/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:236 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/63631d09-f1c5-43a6-98e9-4002ada7949a/volumes/kubernetes.io~projected/kube-api-access-mgllt:{mountpoint:/var/lib/kubelet/pods/63631d09-f1c5-43a6-98e9-4002ada7949a/volumes/kubernetes.io~projected/kube-api-access-mgllt major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c/volumes/kubernetes.io~projected/kube-api-access-t49f8:{mountpoint:/var/lib/kubelet/pods/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c/volumes/kubernetes.io~projected/kube-api-access-t49f8 major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f/volumes/kubernetes.io~projected/kube-api-access-klgh7:{mountpoint:/var/lib/kubelet/pods/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f/volumes/kubernetes.io~projected/kube-api-access-klgh7 major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7dcd4bbc-400c-4fce-b87c-0c018fadecfa/volumes/kubernetes.io~projected/kube-api-access-5jmmp:{mountpoint:/var/lib/kubelet/pods/7dcd4bbc-400c-4fce-b87c-0c018fadecfa/volumes/kubernetes.io~projected/kube-api-access-5jmmp major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7dcd4bbc-400c-4fce-b87c-0c018fadecfa/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/7dcd4bbc-400c-4fce-b87c-0c018fadecfa/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/80a4a0ad-5738-454b-9818-16b544aa5284/volumes/kubernetes.io~projected/kube-api-access-6vnql:{mountpoint:/var/lib/kubelet/pods/80a4a0ad-5738-454b-9818-16b544aa5284/volumes/kubernetes.io~projected/kube-api-access-6vnql major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/97707149-0248-45a0-87d2-7347b960c2db/volumes/kubernetes.io~projected/kube-api-access-nhnsv:{mountpoint:/var/lib/kubelet/pods/97707149-0248-45a0-87d2-7347b960c2db/volumes/kubernetes.io~projected/kube-api-access-nhnsv major:0 minor:345 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/volumes/kubernetes.io~secret/agent-certs major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d085676f-b286-4834-9e62-039f2b6e7f72/volumes/kubernetes.io~projected/kube-api-access-x2wdv:{mountpoint:/var/lib/kubelet/pods/d085676f-b286-4834-9e62-039f2b6e7f72/volumes/kubernetes.io~projected/kube-api-access-x2wdv major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d085676f-b286-4834-9e62-039f2b6e7f72/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/d085676f-b286-4834-9e62-039f2b6e7f72/volumes/kubernetes.io~secret/cert major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f33f6c60-eff2-40c2-9350-ba10eaf5442d/volumes/kubernetes.io~projected/kube-api-access-bqkrc:{mountpoint:/var/lib/kubelet/pods/f33f6c60-eff2-40c2-9350-ba10eaf5442d/volumes/kubernetes.io~projected/kube-api-access-bqkrc major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c/volumes/kubernetes.io~projected/kube-api-access-b2b6n:{mountpoint:/var/lib/kubelet/pods/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c/volumes/kubernetes.io~projected/kube-api-access-b2b6n major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fa9b645b-69d5-47a7-8457-409364209023/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/fa9b645b-69d5-47a7-8457-409364209023/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:340 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/3683b39c1e80e10633c028e8bb10e3b456e99b6f084dc79ef32169ac2cd97ea6/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/ee2bccfa801a34abe66a626a5eaa5ed1f6f9f99ca98f9659efe963f12f56794a/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/3c518f2b110c66a3567a199548796a5703ae443d031bb628189aa472edab2cbd/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/ccb6a3d1f642e9464bcccbe34457387f7b187d44ad4e3dde7bb126d3f523a02d/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-111:{mountpoint:/var/lib/containers/storage/overlay/fa8e6c4662b3d557b902dfd803108769e2f8aded9305cabf6ce152a7817fb108/merged major:0 minor:111 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/6867d8e539646bf9a15d63eb39acd83e08a67bb1cc667a545beb11046329c754/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/fb9180167e24e20d07c0c3c372dfa450ab6da6937a99faed29fe592f769ab565/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/948d76d194923a0c1a05e07420fedc624f483540b9b2b86595f8c5f3065ae8c2/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/de3a68547f7d0efd79e8ba0a036530759276b982ae348a198725ef1402d4d4d9/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/0189d7ae63e878b818c5f2630d94e07749ce904cd0accb89bcd0ba1ddfc2499c/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/8fb7195a2f306ee7f90ebddb18c36b9bcb39f61c2188ec0cf65e4b7d12a90e22/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/dad9351c7343d9a5c93f5bc5d8649d9652d4cb7d1cb4c234460dc9c093549bfc/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/9a7eca42b54146260859f75087453e7caa4e327f06335007b585b28fadb7adcc/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/682e848f7134f9e094e568d4bccc976d81afb9098264da5326974d9f84f3b54e/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/bd7a38ee382b5a87c001cafb7dca3e1962477575f673f88f9fb9ef920ddf59f8/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/1e51bb5733b72eeab7f0719c80bbb56f595b153184fadbbaa57c558947c34ca5/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/23894d2cae2513e15796721a0d79a12b5040925f126bab7b864f67a143900614/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/761900d10598214b4c2f5514450bc27c73896177e7d2dc7b83a0d947bdb1631f/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/b94e8aa45656f91acbfdca4a4dd88e194c461e9c3005171920367c2255e1d4fa/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/68a04301683d261c26fece167562517884b067608285a5ca861c499cfa8b0647/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/e59f44c559bad9435fa95cd4a68f5bb43feb2fd647c7bee1db4dd29558c2f3a1/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/088d0d546a02264e7206eeec42f574b221e8965d35dad13b49c7bdd25a1000d8/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/151a67a2d7ad0355325610343881c71d6faba2c0ae543915a4ecb8bc52b83a41/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/33089dc411eb946ff140ff2708c9caad0629bbd23eea646b3653a63e01cf7f48/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/505cb17d934136029b14369dc357dd7c53c16f57ff985a5f79a62288d07db13b/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-223:{mountpoint:/var/lib/containers/storage/overlay/452cc9ae3de46c9ce903ae731f94466d88b7bb1a3e23e0d371f66b3ad7b5dbdb/merged major:0 minor:223 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/3062374ba0fdd4d91124f490550b660fc7925733ce3571aef52f786d25d3b0e0/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/2f76b7276ea601510356e53fb1bf4ad1cdc23a68e6594511fa02fb308b09f511/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-244:{mountpoint:/var/lib/containers/storage/overlay/fdaaaf479a0ce857829f5d2cfc70d823c828e04e71104b764f9c1a4215dae9bb/merged major:0 minor:244 fsType:overlay blockSize:0} overlay_0-246:{mountpoint:/var/lib/containers/storage/overlay/6d5a2a04e82a607df4b1c7eac70744adab25aae8bed8191c631ec58adbe37740/merged major:0 minor:246 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/777ce419263b899b0f077c3f65c2f47014ebfaf2ea1d1f436eb4b0973dc26a6c/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/846e08ea2c9bb60cc62ac2ea0c72afe3d847b0c99ac8f32ffa98288defd31c95/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-259:{mountpoint:/var/lib/containers/storage/overlay/d0783f42edb2bbaeb65a77a5baa948ec7dfd231abb9d6dd7618aaa94125c26ed/merged major:0 minor:259 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/f1f73e01ec331ff954cf61f53bdd0698ce5485af22484c6aa880803e185bef4c/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-280:{mountpoint:/var/lib/containers/storage/overlay/ff5096b712e823b2efbafb8d9afc2e33ff82b90b59c784f180fe4986526aeb81/merged major:0 minor:280 fsType:overlay blockSize:0} overlay_0-282:{mountpoint:/var/lib/containers/storage/overlay/65fdac595791e06032bdf6bfe6333719c722ad8f07782f1f4f5f87a0cdf1b316/merged major:0 minor:282 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/a365d58270b1936731b017b536c381c43d879e06e630a2b8a32d901b6302f81f/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-301:{mountpoint:/var/lib/containers/storage/overlay/4f58846a621f0f319cfcb902a8506d4340734bffcdd686faf234969c08852127/merged major:0 minor:301 fsType:overlay blockSize:0} overlay_0-303:{mountpoint:/var/lib/containers/storage/overlay/2f09e3131fc1d554673eef2ab2b5ed7a3eb26352d24859907259cdb242915060/merged major:0 minor:303 fsType:overlay blockSize:0} overlay_0-309:{mountpoint:/var/lib/containers/storage/overlay/f7bd87a7c8d28dafd757cc6d2722d47d763a377cad77496e0a955fcb3735658b/merged major:0 minor:309 fsType:overlay blockSize:0} overlay_0-314:{mountpoint:/var/lib/containers/storage/overlay/ed0a1787e4be7324716fce55f154b96ddcddc3784e74b619ff32429cf41220cf/merged major:0 minor:314 fsType:overlay blockSize:0} overlay_0-320:{mountpoint:/var/lib/containers/storage/overlay/087d3e739e4ead25a557b4432b5ec09963f60488c7fb0c79fd339c96c97e854d/merged major:0 minor:320 fsType:overlay blockSize:0} overlay_0-325:{mountpoint:/var/lib/containers/storage/overlay/d6cebb0364cfc51b23966b6e8d1b53fc4dd7cc481280c30e8864a26859f8f676/merged major:0 minor:325 fsType:overlay blockSize:0} overlay_0-330:{mountpoint:/var/lib/containers/storage/overlay/cb5ce347b17d8c61bff50273ecf17363a6171e66dc6c3a5a8cdefbec62330e8c/merged major:0 minor:330 fsType:overlay blockSize:0} overlay_0-335:{mountpoint:/var/lib/containers/storage/overlay/13799332381b885012271aabbc03d74defbe896f509b445c5c18b02deedb18d5/merged major:0 minor:335 fsType:overlay blockSize:0} overlay_0-357:{mountpoint:/var/lib/containers/storage/overlay/0f8cddd8f1387d4afe4a049185238796f0ef433d7c6653aa5bfabf4918fd5478/merged major:0 minor:357 fsType:overlay blockSize:0} overlay_0-359:{mountpoint:/var/lib/containers/storage/overlay/24c52afc0c90e14411a6cdd8af43e496972c77ac335f34566c2f297be714554c/merged major:0 minor:359 fsType:overlay blockSize:0} overlay_0-361:{mountpoint:/var/lib/containers/storage/overlay/350537cfb02857ac1e68b7ce8dac92cba52588f9a67ebe15f961e83aa25c6025/merged major:0 minor:361 fsType:overlay blockSize:0} overlay_0-363:{mountpoint:/var/lib/containers/storage/overlay/90ffb1b68b9057695fee2f0cfb3c61fbf5bd89f1084cc37c7c40260db7265a4a/merged major:0 minor:363 fsType:overlay blockSize:0} overlay_0-365:{mountpoint:/var/lib/containers/storage/overlay/4a80190bfe69462006c219251ebc62edc19edbba066edf5a6d8a653ef7e28842/merged major:0 minor:365 fsType:overlay blockSize:0} overlay_0-367:{mountpoint:/var/lib/containers/storage/overlay/98a22cbe2c046309b27a245f43070769563a976d9d7b30fb0c4cd16f589afa29/merged major:0 minor:367 fsType:overlay blockSize:0} overlay_0-373:{mountpoint:/var/lib/containers/storage/overlay/1359c279c9513e5b9c0053b767f484032ba6b44298f0a3048a142022f831416a/merged major:0 minor:373 fsType:overlay blockSize:0} overlay_0-379:{mountpoint:/var/lib/containers/storage/overlay/121afe7286bd4301750767f2c858722b45516971988c45cbfec0579f4bc2103d/merged major:0 minor:379 fsType:overlay blockSize:0} overlay_0-384:{mountpoint:/var/lib/containers/storage/overlay/78e6d6f52dab3cf2789e86b9479410cd3e3487ff898dd633a68ef896d775f2e0/merged major:0 minor:384 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/123a8d19e9177d227e71c91221c7dd65d27e6a4923e05994538df8ba5e637987/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/e6e1f81dd42e559d85958bee701dcd64688c8008b04b621dcef77d2b7e9543e2/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/c2165b5b82a01aabfaa9fd0098b76406083205ac9883c76376f9924b5bfdd3c5/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/8e269ac96bc6003deab8d42f852f90b783e8820273bfc8d552cd046d47c58589/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/bea41d775dd083d7ff3d1e945afe479f6abb839be82ed7b2f4446414d40b6202/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/f535bd09e10dddb8d4e20f2c78957c71cc54f6c39432bdf7d17004a8f8d99b94/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-85:{mountpoint:/var/lib/containers/storage/overlay/9c1159084b87c47de025caf07db0e7fd1845906e1f0d19637c0fd806b073dc7f/merged major:0 minor:85 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/8cd5dae45c8bf96123399f821de59ab268110daf24af05a5ecb9b731717e9a21/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/7239235a09621687fe4f5e381f1057a9502f2911a0be9e1245ea5d97178e7763/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/5e422e9ae7b0964c92a5bd3cd61bd8ad9956285db21bd1085486563faa559c1a/merged major:0 minor:96 fsType:overlay blockSize:0}] Nov 04 07:16:04.749072 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.748155 5385 manager.go:217] Machine: {Timestamp:2026-11-04 07:16:04.747331109 +0000 UTC m=+0.081270687 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102563 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2de77bb6519bb8ed936be41ce50486 SystemUUID:ec2de77b-b651-9bb8-ed93-6be41ce50486 BootID:9279c8cf-a8da-4075-8505-954e6c7043cd Filesystems:[{Device:/var/lib/kubelet/pods/17916760-be6c-47bf-8569-94ec0e36e304/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:344 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0920f6d09f5542e9ff46a73d1e32487fa63e11fe8640270165d7c37419792f2b/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e1efd0dbd7ffad01b836b1df99461789e63e533d166951a6420a7a5596cb62bf/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:291 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-367 DeviceMajor:0 DeviceMinor:367 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/51538327-34ef-409b-b76a-6963b7254eec/volumes/kubernetes.io~projected/kube-api-access-7m7f6 DeviceMajor:0 DeviceMinor:238 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-259 DeviceMajor:0 DeviceMinor:259 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-309 DeviceMajor:0 DeviceMinor:309 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/709bfc975391d29847dce94faba2126dbd67b2d87a85fc353b0f0245dd41f34e/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c/volumes/kubernetes.io~projected/kube-api-access-b2b6n DeviceMajor:0 DeviceMinor:57 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b158853477c105444b0009f84317f574595aad43b9046ecdb6eed47b76170255/userdata/shm DeviceMajor:0 DeviceMinor:355 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-320 DeviceMajor:0 DeviceMinor:320 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad/volumes/kubernetes.io~projected/kube-api-access-rtzpm DeviceMajor:0 DeviceMinor:64 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/19b2ab56-a868-4570-b098-37c517da236a/volumes/kubernetes.io~projected/kube-api-access-f2dxb DeviceMajor:0 DeviceMinor:67 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/192706c9c89870c90332435973b8d137b0548ea06c810e67817775533b83ccc9/userdata/shm DeviceMajor:0 DeviceMinor:97 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/508fee92-08e3-49ab-8b16-3add538b11eb/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:197 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-325 DeviceMajor:0 DeviceMinor:325 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-280 DeviceMajor:0 DeviceMinor:280 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/97707149-0248-45a0-87d2-7347b960c2db/volumes/kubernetes.io~projected/kube-api-access-nhnsv DeviceMajor:0 DeviceMinor:345 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4c6a17b097876c7082ce87b31d331614041295afbbb1e5d1d28f674afc0e94b7/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/d085676f-b286-4834-9e62-039f2b6e7f72/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:203 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b863cc9011c198aadf7cc1bf200574267b9da8b94ddfc802cd35e134a423fd52/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:292 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2a0c856154e6e0535c5bc1b0fc56fd5269d08bac276190c963ee0aa3f660c6f4/userdata/shm DeviceMajor:0 DeviceMinor:300 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e831a99c631e426e454e7baf13c0458c225215cf935034bd771343ed086fc2c4/userdata/shm DeviceMajor:0 DeviceMinor:83 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/7dcd4bbc-400c-4fce-b87c-0c018fadecfa/volumes/kubernetes.io~projected/kube-api-access-5jmmp DeviceMajor:0 DeviceMinor:202 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/562fe41876ab323191b043ffbc10fa295c9d3b5b3c3f24ff8919854c6be94834/userdata/shm DeviceMajor:0 DeviceMinor:278 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-282 DeviceMajor:0 DeviceMinor:282 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1172079a-2931-4ecb-9196-68e6da14fd36/volumes/kubernetes.io~projected/kube-api-access-hskwv DeviceMajor:0 DeviceMinor:270 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:290 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:296 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-357 DeviceMajor:0 DeviceMinor:357 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:295 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~projected/kube-api-access-cqrrw DeviceMajor:0 DeviceMinor:297 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-361 DeviceMajor:0 DeviceMinor:361 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7dcd4bbc-400c-4fce-b87c-0c018fadecfa/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:45 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/684c4196729a5a5c95e2abb153f5df3b2a2c0f984ecc4c757a346a9f663b5e7a/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1172079a-2931-4ecb-9196-68e6da14fd36/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:269 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fc16b866b67f1e6c70bb4ee91fdc2a6fb1b7120131aaaee82fb4f1b3a30bcc0d/userdata/shm DeviceMajor:0 DeviceMinor:271 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-314 DeviceMajor:0 DeviceMinor:314 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/30df6026-1baf-4fca-8c94-cb092397a676/volumes/kubernetes.io~projected/kube-api-access-tzs4d DeviceMajor:0 DeviceMinor:348 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:62 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-246 DeviceMajor:0 DeviceMinor:246 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:293 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-384 DeviceMajor:0 DeviceMinor:384 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1488becb-97bc-43e6-b85c-6c9c62dfe903/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:275 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-365 DeviceMajor:0 DeviceMinor:365 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/80a4a0ad-5738-454b-9818-16b544aa5284/volumes/kubernetes.io~projected/kube-api-access-6vnql DeviceMajor:0 DeviceMinor:60 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1172079a-2931-4ecb-9196-68e6da14fd36/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:265 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-363 DeviceMajor:0 DeviceMinor:363 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-373 DeviceMajor:0 DeviceMinor:373 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f/volumes/kubernetes.io~projected/kube-api-access-klgh7 DeviceMajor:0 DeviceMinor:61 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/51538327-34ef-409b-b76a-6963b7254eec/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:236 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/51538327-34ef-409b-b76a-6963b7254eec/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:237 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:290 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:290 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-335 DeviceMajor:0 DeviceMinor:335 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fa9b645b-69d5-47a7-8457-409364209023/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:340 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-359 DeviceMajor:0 DeviceMinor:359 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1488becb-97bc-43e6-b85c-6c9c62dfe903/volumes/kubernetes.io~projected/kube-api-access-wlj2h DeviceMajor:0 DeviceMinor:277 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/30df6026-1baf-4fca-8c94-cb092397a676/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:347 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-303 DeviceMajor:0 DeviceMinor:303 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c3575da54d6411a09114faffa4f53a98f500552978ba887b705138e9eea75446/userdata/shm DeviceMajor:0 DeviceMinor:349 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/run/containers/storage/overlay-containers/74dde8637d7b8fd440905c117fd8e135e7a24ffb9b04c353c7920db3f179b5b2/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/63631d09-f1c5-43a6-98e9-4002ada7949a/volumes/kubernetes.io~projected/kube-api-access-mgllt DeviceMajor:0 DeviceMinor:65 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5b287a145f727b1a4a1a27b9710ee3ff2ed4cf8a7797b20a959e36933e317fc8/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/42519616ec898f3cc24635c600c6319af9749812f96791d88ba1f4dbd5823264/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1488becb-97bc-43e6-b85c-6c9c62dfe903/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:276 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c/volumes/kubernetes.io~projected/kube-api-access-t49f8 DeviceMajor:0 DeviceMinor:59 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-111 DeviceMajor:0 DeviceMinor:111 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:289 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5fc708252796083f69ed7f8766d4018af524ce264be29bba95ae7dac9aa83d4e/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/508fee92-08e3-49ab-8b16-3add538b11eb/volumes/kubernetes.io~projected/kube-api-access-j4rg8 DeviceMajor:0 DeviceMinor:201 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/d085676f-b286-4834-9e62-039f2b6e7f72/volumes/kubernetes.io~projected/kube-api-access-x2wdv DeviceMajor:0 DeviceMinor:204 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-223 DeviceMajor:0 DeviceMinor:223 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-301 DeviceMajor:0 DeviceMinor:301 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-379 DeviceMajor:0 DeviceMinor:379 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a2a9454ff0a9aaee20e782323e0e1f9f2db0f10bed694f3003567300d327e1ec/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:63 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:294 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-330 DeviceMajor:0 DeviceMinor:330 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/30df6026-1baf-4fca-8c94-cb092397a676/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:346 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4df9c74ae689e955ce0d7e956d9371ee0f78516b2d94770eb563f4266120b0ca/userdata/shm DeviceMajor:0 DeviceMinor:351 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:290 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4f9d24ff5fa1251fad621af3e696722792385fda408c20e827e43db5e133ec3b/userdata/shm DeviceMajor:0 DeviceMinor:239 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7690cf4f07ef529c2acc86e76277753fc1f838a5cbbdf28ff4176f47a487c129/userdata/shm DeviceMajor:0 DeviceMinor:353 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/f33f6c60-eff2-40c2-9350-ba10eaf5442d/volumes/kubernetes.io~projected/kube-api-access-bqkrc DeviceMajor:0 DeviceMinor:58 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:55 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-85 DeviceMajor:0 DeviceMinor:85 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-244 DeviceMajor:0 DeviceMinor:244 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:284 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/17916760-be6c-47bf-8569-94ec0e36e304/volumes/kubernetes.io~projected/kube-api-access-lsljv DeviceMajor:0 DeviceMinor:66 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:291 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:2a0c856154e6e05 MacAddress:c2:9e:61:a4:f9:62 Speed:10000 Mtu:8901} {Name:4df9c74ae689e95 MacAddress:6a:7e:99:3a:69:4f Speed:10000 Mtu:8901} {Name:4f9d24ff5fa1251 MacAddress:aa:02:08:ae:6c:56 Speed:10000 Mtu:8901} {Name:684c4196729a5a5 MacAddress:6a:17:48:3c:ff:96 Speed:10000 Mtu:8901} {Name:7690cf4f07ef529 MacAddress:8e:59:c9:8d:3d:61 Speed:10000 Mtu:8901} {Name:b158853477c1054 MacAddress:16:c1:b3:b6:44:ac Speed:10000 Mtu:8901} {Name:b863cc9011c198a MacAddress:3a:65:a8:a7:55:15 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:d1:c3:80:4b Speed:0 Mtu:9001} {Name:br-int MacAddress:12:28:c3:ce:d1:e3 Speed:0 Mtu:8901} {Name:c3575da54d6411a MacAddress:a6:4b:7f:92:1d:75 Speed:10000 Mtu:8901} {Name:e1efd0dbd7ffad0 MacAddress:8e:00:4d:07:d4:57 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:d1:c3:80:4b Speed:0 Mtu:9001} {Name:fc16b866b67f1e6 MacAddress:e6:f5:38:bf:e8:0a Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:5a:b4:1c:5d:24:4d Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:d6:38:a6:f2:69:01 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 07:16:04.749356 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.749080 5385 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 07:16:04.749356 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.749144 5385 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 07:16:04.749590 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.749550 5385 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 07:16:04.750109 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.749592 5385 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-6.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 07:16:04.750192 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.750113 5385 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 07:16:04.750192 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.750132 5385 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 07:16:04.750192 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.750163 5385 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 07:16:04.750286 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.750192 5385 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 07:16:04.750344 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.750333 5385 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 07:16:04.750582 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.750571 5385 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 07:16:04.750699 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.750690 5385 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 07:16:04.750732 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.750727 5385 kubelet.go:493] "Attempting to sync node with API server" Nov 04 07:16:04.750760 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.750738 5385 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 07:16:04.750760 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.750748 5385 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 07:16:04.750760 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.750756 5385 kubelet.go:405] "Adding apiserver pod source" Nov 04 07:16:04.750861 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.750766 5385 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 07:16:04.751519 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.751485 5385 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 07:16:04.751519 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.751500 5385 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 07:16:04.752002 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.751983 5385 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 07:16:04.755300 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755283 5385 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 07:16:04.755300 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755297 5385 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 07:16:04.755739 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755409 5385 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Nov 04 07:16:04.755739 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755667 5385 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 07:16:04.755739 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755696 5385 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 07:16:04.755868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755794 5385 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 07:16:04.755868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755807 5385 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 07:16:04.755868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755813 5385 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 07:16:04.755868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755820 5385 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 07:16:04.755868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755829 5385 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 07:16:04.755868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755837 5385 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 07:16:04.755868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755843 5385 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 07:16:04.755868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755849 5385 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 07:16:04.755868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755856 5385 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 07:16:04.755868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755862 5385 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 07:16:04.755868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755871 5385 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 07:16:04.756251 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755882 5385 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 07:16:04.756251 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755906 5385 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 07:16:04.756251 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.755911 5385 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 07:16:04.759327 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.759311 5385 server.go:1263] "Started kubelet" Nov 04 07:16:04.759451 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.759400 5385 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 07:16:04.765463 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.759621 5385 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 07:16:04.765463 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.759688 5385 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 07:16:04.765463 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.760002 5385 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 07:16:04.765463 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.760851 5385 server.go:317] "Adding debug handlers to kubelet server" Nov 04 07:16:04.765463 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.763232 5385 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 07:16:04.765463 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.763472 5385 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 07:16:04.765463 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.764872 5385 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 07:09:55 +0000 UTC" deadline="2028-04-25 03:38:03.698947813 +0000 UTC" Nov 04 07:16:04.765463 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.765034 5385 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12908h21m58.933917792s" Nov 04 07:16:04.765463 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.765073 5385 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 07:16:04.765463 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.765088 5385 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 07:16:04.765463 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.765203 5385 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 07:16:04.759894 ip-10-0-131-6 systemd[1]: Started Kubernetes Kubelet. Nov 04 07:16:04.766098 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:04.765643 5385 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-6.us-east-2.compute.internal\" not found" Nov 04 07:16:04.769014 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.768957 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="80a4a0ad-5738-454b-9818-16b544aa5284" volumeName="kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-cni-binary-copy" seLinuxMountContext="" Nov 04 07:16:04.769107 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769040 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="80a4a0ad-5738-454b-9818-16b544aa5284" volumeName="kubernetes.io/projected/80a4a0ad-5738-454b-9818-16b544aa5284-kube-api-access-6vnql" seLinuxMountContext="" Nov 04 07:16:04.769107 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769074 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="97707149-0248-45a0-87d2-7347b960c2db" volumeName="kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv" seLinuxMountContext="" Nov 04 07:16:04.769107 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769093 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1172079a-2931-4ecb-9196-68e6da14fd36" volumeName="kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-console-config" seLinuxMountContext="" Nov 04 07:16:04.769107 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769105 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30ff434f-8151-47d7-b6e9-51d9a3d8d6ad" volumeName="kubernetes.io/secret/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovn-node-metrics-cert" seLinuxMountContext="" Nov 04 07:16:04.769320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769127 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3632b027-6f63-4015-a775-b3a7d6d32247" volumeName="kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-main-tls" seLinuxMountContext="" Nov 04 07:16:04.769320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769146 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1488becb-97bc-43e6-b85c-6c9c62dfe903" volumeName="kubernetes.io/configmap/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-accelerators-collector-config" seLinuxMountContext="" Nov 04 07:16:04.769320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769170 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="508fee92-08e3-49ab-8b16-3add538b11eb" volumeName="kubernetes.io/empty-dir/508fee92-08e3-49ab-8b16-3add538b11eb-tmp-dir" seLinuxMountContext="" Nov 04 07:16:04.769320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769201 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51538327-34ef-409b-b76a-6963b7254eec" volumeName="kubernetes.io/secret/51538327-34ef-409b-b76a-6963b7254eec-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext="" Nov 04 07:16:04.769320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769214 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f33f6c60-eff2-40c2-9350-ba10eaf5442d" volumeName="kubernetes.io/projected/f33f6c60-eff2-40c2-9350-ba10eaf5442d-kube-api-access-bqkrc" seLinuxMountContext="" Nov 04 07:16:04.769320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769250 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1488becb-97bc-43e6-b85c-6c9c62dfe903" volumeName="kubernetes.io/secret/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-kube-rbac-proxy-config" seLinuxMountContext="" Nov 04 07:16:04.769320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769274 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30ff434f-8151-47d7-b6e9-51d9a3d8d6ad" volumeName="kubernetes.io/projected/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-kube-api-access-rtzpm" seLinuxMountContext="" Nov 04 07:16:04.769320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769287 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3632b027-6f63-4015-a775-b3a7d6d32247" volumeName="kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-web-config" seLinuxMountContext="" Nov 04 07:16:04.769764 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769333 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="508fee92-08e3-49ab-8b16-3add538b11eb" volumeName="kubernetes.io/configmap/508fee92-08e3-49ab-8b16-3add538b11eb-config-volume" seLinuxMountContext="" Nov 04 07:16:04.769764 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769363 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51538327-34ef-409b-b76a-6963b7254eec" volumeName="kubernetes.io/configmap/51538327-34ef-409b-b76a-6963b7254eec-metrics-client-ca" seLinuxMountContext="" Nov 04 07:16:04.769764 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769379 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a43577d-3b09-4f31-89ec-5b4b7f7efb4c" volumeName="kubernetes.io/projected/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-kube-api-access-t49f8" seLinuxMountContext="" Nov 04 07:16:04.769764 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769390 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fa408a9f-2280-4a77-bddd-c31dbe7fdb9c" volumeName="kubernetes.io/projected/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-kube-api-access-b2b6n" seLinuxMountContext="" Nov 04 07:16:04.769764 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769417 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19b2ab56-a868-4570-b098-37c517da236a" volumeName="kubernetes.io/projected/19b2ab56-a868-4570-b098-37c517da236a-kube-api-access-f2dxb" seLinuxMountContext="" Nov 04 07:16:04.769764 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769432 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30df6026-1baf-4fca-8c94-cb092397a676" volumeName="kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-oauth-serving-cert" seLinuxMountContext="" Nov 04 07:16:04.769764 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769457 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="508fee92-08e3-49ab-8b16-3add538b11eb" volumeName="kubernetes.io/projected/508fee92-08e3-49ab-8b16-3add538b11eb-kube-api-access-j4rg8" seLinuxMountContext="" Nov 04 07:16:04.769764 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769475 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3632b027-6f63-4015-a775-b3a7d6d32247" volumeName="kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-metrics-client-ca" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.769489 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3632b027-6f63-4015-a775-b3a7d6d32247" volumeName="kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.770603 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a43577d-3b09-4f31-89ec-5b4b7f7efb4c" volumeName="kubernetes.io/empty-dir/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-tuned" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.770633 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1488becb-97bc-43e6-b85c-6c9c62dfe903" volumeName="kubernetes.io/empty-dir/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-textfile" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.770660 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1488becb-97bc-43e6-b85c-6c9c62dfe903" volumeName="kubernetes.io/projected/1488becb-97bc-43e6-b85c-6c9c62dfe903-kube-api-access-wlj2h" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.770703 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17916760-be6c-47bf-8569-94ec0e36e304" volumeName="kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.770726 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30df6026-1baf-4fca-8c94-cb092397a676" volumeName="kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-trusted-ca-bundle" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.770742 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7dcd4bbc-400c-4fce-b87c-0c018fadecfa" volumeName="kubernetes.io/secret/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-insights-runtime-extractor-tls" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.770766 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="80a4a0ad-5738-454b-9818-16b544aa5284" volumeName="kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-whereabouts-flatfile-configmap" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.770794 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30df6026-1baf-4fca-8c94-cb092397a676" volumeName="kubernetes.io/projected/30df6026-1baf-4fca-8c94-cb092397a676-kube-api-access-tzs4d" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.770863 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30ff434f-8151-47d7-b6e9-51d9a3d8d6ad" volumeName="kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-env-overrides" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.770905 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3632b027-6f63-4015-a775-b3a7d6d32247" volumeName="kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-kube-api-access-cqrrw" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.770918 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7dcd4bbc-400c-4fce-b87c-0c018fadecfa" volumeName="kubernetes.io/empty-dir/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-data-volume" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.770934 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d085676f-b286-4834-9e62-039f2b6e7f72" volumeName="kubernetes.io/secret/d085676f-b286-4834-9e62-039f2b6e7f72-cert" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.770960 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1172079a-2931-4ecb-9196-68e6da14fd36" volumeName="kubernetes.io/empty-dir/1172079a-2931-4ecb-9196-68e6da14fd36-tmp" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771002 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1172079a-2931-4ecb-9196-68e6da14fd36" volumeName="kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-oauth-config" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771018 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="19b2ab56-a868-4570-b098-37c517da236a" volumeName="kubernetes.io/empty-dir/19b2ab56-a868-4570-b098-37c517da236a-tmp-dir" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771044 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30df6026-1baf-4fca-8c94-cb092397a676" volumeName="kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-oauth-config" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771077 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3632b027-6f63-4015-a775-b3a7d6d32247" volumeName="kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-trusted-ca-bundle" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771093 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6a43577d-3b09-4f31-89ec-5b4b7f7efb4c" volumeName="kubernetes.io/empty-dir/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-tmp" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771173 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1488becb-97bc-43e6-b85c-6c9c62dfe903" volumeName="kubernetes.io/configmap/1488becb-97bc-43e6-b85c-6c9c62dfe903-metrics-client-ca" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771218 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1488becb-97bc-43e6-b85c-6c9c62dfe903" volumeName="kubernetes.io/secret/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-tls" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771261 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="508fee92-08e3-49ab-8b16-3add538b11eb" volumeName="kubernetes.io/secret/508fee92-08e3-49ab-8b16-3add538b11eb-metrics-tls" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771296 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51538327-34ef-409b-b76a-6963b7254eec" volumeName="kubernetes.io/secret/51538327-34ef-409b-b76a-6963b7254eec-prometheus-operator-tls" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771313 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="80a4a0ad-5738-454b-9818-16b544aa5284" volumeName="kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-cni-sysctl-allowlist" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771325 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9" volumeName="kubernetes.io/configmap/ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9-konnectivity-ca" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771622 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51538327-34ef-409b-b76a-6963b7254eec" volumeName="kubernetes.io/projected/51538327-34ef-409b-b76a-6963b7254eec-kube-api-access-7m7f6" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771666 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f" volumeName="kubernetes.io/projected/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-kube-api-access-klgh7" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771713 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1172079a-2931-4ecb-9196-68e6da14fd36" volumeName="kubernetes.io/projected/1172079a-2931-4ecb-9196-68e6da14fd36-kube-api-access-hskwv" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771741 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1172079a-2931-4ecb-9196-68e6da14fd36" volumeName="kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-serving-cert" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771773 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30df6026-1baf-4fca-8c94-cb092397a676" volumeName="kubernetes.io/empty-dir/30df6026-1baf-4fca-8c94-cb092397a676-tmp" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771805 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30df6026-1baf-4fca-8c94-cb092397a676" volumeName="kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-serving-cert" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771828 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30ff434f-8151-47d7-b6e9-51d9a3d8d6ad" volumeName="kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovnkube-script-lib" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771847 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3632b027-6f63-4015-a775-b3a7d6d32247" volumeName="kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-main-db" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771873 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3632b027-6f63-4015-a775-b3a7d6d32247" volumeName="kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771897 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7dcd4bbc-400c-4fce-b87c-0c018fadecfa" volumeName="kubernetes.io/projected/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-kube-api-access-5jmmp" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.771990 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9" volumeName="kubernetes.io/secret/ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9-agent-certs" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772012 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d085676f-b286-4834-9e62-039f2b6e7f72" volumeName="kubernetes.io/projected/d085676f-b286-4834-9e62-039f2b6e7f72-kube-api-access-x2wdv" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772054 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fa9b645b-69d5-47a7-8457-409364209023" volumeName="kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772078 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3632b027-6f63-4015-a775-b3a7d6d32247" volumeName="kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-config-volume" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772092 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f" volumeName="kubernetes.io/configmap/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-cni-binary-copy" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772117 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f" volumeName="kubernetes.io/configmap/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-daemon-config" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772132 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f33f6c60-eff2-40c2-9350-ba10eaf5442d" volumeName="kubernetes.io/configmap/f33f6c60-eff2-40c2-9350-ba10eaf5442d-serviceca" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772248 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1172079a-2931-4ecb-9196-68e6da14fd36" volumeName="kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-oauth-serving-cert" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772265 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1172079a-2931-4ecb-9196-68e6da14fd36" volumeName="kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-service-ca" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772277 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30df6026-1baf-4fca-8c94-cb092397a676" volumeName="kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-service-ca" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772305 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3632b027-6f63-4015-a775-b3a7d6d32247" volumeName="kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-tls-assets" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772319 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="17916760-be6c-47bf-8569-94ec0e36e304" volumeName="kubernetes.io/projected/17916760-be6c-47bf-8569-94ec0e36e304-kube-api-access-lsljv" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772363 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3632b027-6f63-4015-a775-b3a7d6d32247" volumeName="kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-config-out" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772375 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3632b027-6f63-4015-a775-b3a7d6d32247" volumeName="kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-cluster-tls-config" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772387 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3632b027-6f63-4015-a775-b3a7d6d32247" volumeName="kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772402 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63631d09-f1c5-43a6-98e9-4002ada7949a" volumeName="kubernetes.io/configmap/63631d09-f1c5-43a6-98e9-4002ada7949a-iptables-alerter-script" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772469 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30df6026-1baf-4fca-8c94-cb092397a676" volumeName="kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-console-config" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772490 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="30ff434f-8151-47d7-b6e9-51d9a3d8d6ad" volumeName="kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovnkube-config" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772660 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63631d09-f1c5-43a6-98e9-4002ada7949a" volumeName="kubernetes.io/projected/63631d09-f1c5-43a6-98e9-4002ada7949a-kube-api-access-mgllt" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772673 5385 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7dcd4bbc-400c-4fce-b87c-0c018fadecfa" volumeName="kubernetes.io/configmap/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-kube-rbac-proxy-cm" seLinuxMountContext="" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772687 5385 reconstruct.go:97] "Volume reconstruction finished" Nov 04 07:16:04.773525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.772701 5385 reconciler.go:29] "Reconciler: start to sync state" Nov 04 07:16:04.776863 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.773757 5385 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 07:16:04.776863 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.773767 5385 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 07:16:04.776863 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.773781 5385 factory.go:55] Registering systemd factory Nov 04 07:16:04.776863 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.773788 5385 factory.go:223] Registration of the systemd container factory successfully Nov 04 07:16:04.780228 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.780207 5385 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 07:16:04.780395 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.780380 5385 factory.go:153] Registering CRI-O factory Nov 04 07:16:04.780442 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.780400 5385 factory.go:223] Registration of the crio container factory successfully Nov 04 07:16:04.780442 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.780418 5385 factory.go:103] Registering Raw factory Nov 04 07:16:04.780442 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.780431 5385 manager.go:1199] Started watching for new ooms in manager Nov 04 07:16:04.781444 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.780700 5385 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Nov 04 07:16:04.781444 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.781317 5385 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 07:16:04.782276 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.782259 5385 manager.go:319] Starting recovery of all containers Nov 04 07:16:04.805098 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.803366 5385 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 07:16:04.805098 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.804806 5385 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 07:16:04.805098 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.804823 5385 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 07:16:04.805098 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.804842 5385 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 07:16:04.805098 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:04.804882 5385 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Nov 04 07:16:04.808846 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.808818 5385 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 07:16:04.822968 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.822932 5385 generic.go:356] "Generic (PLEG): container finished" podID="b58482bdf311c221bced295e67dfb26d" containerID="f5208030fa712169bbac43c893eb74643314eda131eb62333e9ef8bc83bf153b" exitCode=0 Nov 04 07:16:04.834117 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.834089 5385 generic.go:356] "Generic (PLEG): container finished" podID="3632b027-6f63-4015-a775-b3a7d6d32247" containerID="5782cea40175ed4b81900890fb9483ae7a3eccec289a6f13796e32865641f9ef" exitCode=0 Nov 04 07:16:04.852100 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.852072 5385 generic.go:356] "Generic (PLEG): container finished" podID="80a4a0ad-5738-454b-9818-16b544aa5284" containerID="824f39ef40be88347624d0f709aa880743eb5f54a48113fcc7854259aed4cb3f" exitCode=0 Nov 04 07:16:04.852100 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.852096 5385 generic.go:356] "Generic (PLEG): container finished" podID="80a4a0ad-5738-454b-9818-16b544aa5284" containerID="cbe8dad75dfc4571687cbff1f83d8654f69ce08cdb4be70dedccdda342525b72" exitCode=0 Nov 04 07:16:04.852100 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.852103 5385 generic.go:356] "Generic (PLEG): container finished" podID="80a4a0ad-5738-454b-9818-16b544aa5284" containerID="cf1d9f277fec1a4d634e1a8e2521c9a60f2d349a276a1e17f2f24b16ec67d52a" exitCode=0 Nov 04 07:16:04.852304 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.852109 5385 generic.go:356] "Generic (PLEG): container finished" podID="80a4a0ad-5738-454b-9818-16b544aa5284" containerID="283569bdc3646cf491be01540a163c5e4a380943789d0fc2d26dae67134f6f4e" exitCode=0 Nov 04 07:16:04.852304 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.852115 5385 generic.go:356] "Generic (PLEG): container finished" podID="80a4a0ad-5738-454b-9818-16b544aa5284" containerID="064eeee9c05f1fc7df16e2133423235f6c14b7e4dd6732a0d09c142432f857a1" exitCode=0 Nov 04 07:16:04.852304 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.852120 5385 generic.go:356] "Generic (PLEG): container finished" podID="80a4a0ad-5738-454b-9818-16b544aa5284" containerID="9588d795a7ddd92e5ad48127061801edc96b481a3bde7f723ce0cf9b538d804a" exitCode=0 Nov 04 07:16:04.858585 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.858559 5385 generic.go:356] "Generic (PLEG): container finished" podID="1488becb-97bc-43e6-b85c-6c9c62dfe903" containerID="a557e908a2b85aedd8fa6a97021d5833f02b97cb3d0c63e391b1179aec8c3ee3" exitCode=0 Nov 04 07:16:04.886234 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.886214 5385 manager.go:324] Recovery completed Nov 04 07:16:04.905199 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:04.905161 5385 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Nov 04 07:16:04.912195 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.912173 5385 cpu_manager.go:226] "Starting" policy="none" Nov 04 07:16:04.912195 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.912191 5385 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 07:16:04.912352 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.912213 5385 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 07:16:04.912423 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.912372 5385 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Nov 04 07:16:04.912423 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.912388 5385 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Nov 04 07:16:04.912423 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.912412 5385 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Nov 04 07:16:04.912423 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.912421 5385 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Nov 04 07:16:04.912661 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.912430 5385 policy_none.go:50] "Start" Nov 04 07:16:04.912661 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.912442 5385 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 07:16:04.912661 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.912455 5385 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 07:16:04.912661 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.912658 5385 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Nov 04 07:16:04.912852 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.912670 5385 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Nov 04 07:16:04.912852 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.912678 5385 policy_none.go:44] "Start" Nov 04 07:16:04.917948 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.917913 5385 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 07:16:04.923280 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.923261 5385 manager.go:341] "Starting Device Plugin manager" Nov 04 07:16:04.923791 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:04.923291 5385 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 07:16:04.923878 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.923793 5385 server.go:87] "Starting device plugin registration server" Nov 04 07:16:04.924058 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.924047 5385 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 07:16:04.924101 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.924059 5385 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 07:16:04.924204 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.924155 5385 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 07:16:04.924275 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.924246 5385 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 07:16:04.924378 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.924279 5385 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 07:16:04.924378 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.924286 5385 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 07:16:04.924489 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.924334 5385 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T07:16:04.924269815Z","UUID":"fcddcc12-ca8e-4e8e-a1d5-6521e7b6e481","Handler":null,"Name":"","Endpoint":""} Nov 04 07:16:04.926200 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.926177 5385 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 07:16:04.926347 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:04.926217 5385 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 07:16:04.926445 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:04.926419 5385 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 07:16:05.025014 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.024979 5385 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 07:16:05.027096 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.027066 5385 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasSufficientMemory" Nov 04 07:16:05.027245 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.027103 5385 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasNoDiskPressure" Nov 04 07:16:05.027245 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.027115 5385 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-6.us-east-2.compute.internal" event="NodeHasSufficientPID" Nov 04 07:16:05.027245 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.027160 5385 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.044852 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.044825 5385 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.044974 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.044902 5385 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.105448 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.105383 5385 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal"] Nov 04 07:16:05.106038 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.105996 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" event={"ID":"b58482bdf311c221bced295e67dfb26d","Type":"ContainerStarted","Data":"73b30ee9b7da7a34be4a34e34e562bde1a77ac6d2e6a095991ffae320b9c8c11"} Nov 04 07:16:05.106038 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.106040 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" event={"ID":"b58482bdf311c221bced295e67dfb26d","Type":"ContainerDied","Data":"f5208030fa712169bbac43c893eb74643314eda131eb62333e9ef8bc83bf153b"} Nov 04 07:16:05.106185 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.106060 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" event={"ID":"b58482bdf311c221bced295e67dfb26d","Type":"ContainerStarted","Data":"4c6a17b097876c7082ce87b31d331614041295afbbb1e5d1d28f674afc0e94b7"} Nov 04 07:16:05.106185 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.106107 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" event={"ID":"b7ad388a799a062d464456d3dcd4ae6a","Type":"ContainerStarted","Data":"3c255199ff55b8522bbb1ec55cadec2ea05872ba792c597e9e11b2f9fa9605c1"} Nov 04 07:16:05.106185 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.106125 5385 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.106276 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.106126 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" event={"ID":"b7ad388a799a062d464456d3dcd4ae6a","Type":"ContainerStarted","Data":"74dde8637d7b8fd440905c117fd8e135e7a24ffb9b04c353c7920db3f179b5b2"} Nov 04 07:16:05.106314 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.106206 5385 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.121870 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.121838 5385 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 07:16:05.121988 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:05.121890 5385 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.121988 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.121839 5385 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 07:16:05.121988 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:05.121971 5385 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.183117 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.183033 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b58482bdf311c221bced295e67dfb26d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b58482bdf311c221bced295e67dfb26d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.183117 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.183075 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b58482bdf311c221bced295e67dfb26d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b58482bdf311c221bced295e67dfb26d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.183285 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.183154 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7ad388a799a062d464456d3dcd4ae6a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b7ad388a799a062d464456d3dcd4ae6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.284211 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.284173 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b58482bdf311c221bced295e67dfb26d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b58482bdf311c221bced295e67dfb26d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.284211 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.284211 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b58482bdf311c221bced295e67dfb26d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b58482bdf311c221bced295e67dfb26d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.284413 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.284231 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7ad388a799a062d464456d3dcd4ae6a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b7ad388a799a062d464456d3dcd4ae6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.284413 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.284279 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7ad388a799a062d464456d3dcd4ae6a-config\") pod \"kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b7ad388a799a062d464456d3dcd4ae6a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.284413 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.284286 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b58482bdf311c221bced295e67dfb26d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b58482bdf311c221bced295e67dfb26d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.284413 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.284292 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b58482bdf311c221bced295e67dfb26d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal\" (UID: \"b58482bdf311c221bced295e67dfb26d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.751777 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.751747 5385 apiserver.go:52] "Watching apiserver" Nov 04 07:16:05.766408 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.766379 5385 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 07:16:05.767104 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.767078 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-8656d77c48-qkjzp","openshift-network-diagnostics/network-check-target-9qgxt","openshift-monitoring/node-exporter-ktbsj","openshift-dns/node-resolver-x6v9g","openshift-insights/insights-runtime-extractor-g6lhj","openshift-multus/multus-additional-cni-plugins-grc4v","kube-system/global-pull-secret-syncer-m7gt8","openshift-ovn-kubernetes/ovnkube-node-95dnd","openshift-image-registry/node-ca-sd2bs","openshift-cluster-node-tuning-operator/tuned-dnvwf","openshift-multus/network-metrics-daemon-ldwvk","openshift-console/console-d75fb8d4b-tj7rv","openshift-dns/dns-default-qpt88","openshift-ingress-canary/ingress-canary-pksw6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal","openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5","openshift-network-operator/iptables-alerter-s8gfp","openshift-multus/multus-6jbrj","kube-system/konnectivity-agent-2s2xx","kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal","openshift-monitoring/alertmanager-main-0","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl"] Nov 04 07:16:05.771886 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.771864 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-gnthh\"" Nov 04 07:16:05.772004 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.771983 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 07:16:05.772119 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.772106 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 07:16:05.772438 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.772415 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 07:16:05.772438 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.772428 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 07:16:05.772601 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.772431 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 07:16:05.772601 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.772465 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-l4l9x\"" Nov 04 07:16:05.772601 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.772479 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 07:16:05.774552 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.774532 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 07:16:05.774711 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.774692 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8bggl\"" Nov 04 07:16:05.775095 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.775073 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ltv7r\"" Nov 04 07:16:05.775095 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.775091 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 07:16:05.775378 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.775091 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 07:16:05.775378 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.775236 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 07:16:05.775378 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.775269 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 07:16:05.775378 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.775284 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 07:16:05.775378 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.775338 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-tjts4\"" Nov 04 07:16:05.775647 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.775397 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 07:16:05.775647 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.775474 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 07:16:05.775647 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.775488 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 07:16:05.776428 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.776403 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-slv86\"" Nov 04 07:16:05.776546 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.776465 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-zw52p\"" Nov 04 07:16:05.776919 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.776900 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 07:16:05.778052 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.778000 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 07:16:05.778052 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.778014 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 07:16:05.778052 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.778012 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 07:16:05.778196 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.778058 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7qf26\"" Nov 04 07:16:05.778414 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.778402 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 07:16:05.779479 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.779465 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 07:16:05.779862 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.779850 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 07:16:05.779965 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.779954 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-8kctk\"" Nov 04 07:16:05.781101 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.781082 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 07:16:05.781573 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.781559 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-hvg8n\"" Nov 04 07:16:05.781638 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.781573 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 04 07:16:05.782130 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.782100 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 07:16:05.782130 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.782104 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 07:16:05.782268 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.782249 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 07:16:05.782539 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.782518 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Nov 04 07:16:05.782630 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.782551 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 04 07:16:05.782630 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.782551 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 07:16:05.782630 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.782566 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6pjn4\"" Nov 04 07:16:05.782630 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.782572 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 04 07:16:05.782630 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.782625 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-7rfcz\"" Nov 04 07:16:05.782948 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.782933 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 04 07:16:05.783102 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.783087 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 07:16:05.783138 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.783088 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 07:16:05.783367 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.783344 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tqngx\"" Nov 04 07:16:05.783367 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.783367 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 04 07:16:05.783495 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.783345 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Nov 04 07:16:05.783495 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.783345 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-9lks7\"" Nov 04 07:16:05.783495 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.783377 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 04 07:16:05.783495 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.783372 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 07:16:05.783495 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.783407 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 04 07:16:05.784246 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.784142 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gcxz6\"" Nov 04 07:16:05.784246 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.784160 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Nov 04 07:16:05.784246 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.784173 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Nov 04 07:16:05.784246 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.784185 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 04 07:16:05.784246 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.784190 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 07:16:05.784246 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.784193 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 04 07:16:05.784246 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.784194 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 04 07:16:05.784246 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.784221 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-8zskt\"" Nov 04 07:16:05.784246 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.784196 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 04 07:16:05.784246 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.784229 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Nov 04 07:16:05.784246 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.784245 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-l2xtx\"" Nov 04 07:16:05.784246 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.784249 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mgp2g\"" Nov 04 07:16:05.784630 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.784293 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Nov 04 07:16:05.784917 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.784904 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-h4fsf\"" Nov 04 07:16:05.784962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.784933 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Nov 04 07:16:05.785183 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.785164 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Nov 04 07:16:05.785244 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.785189 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Nov 04 07:16:05.785244 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.785190 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Nov 04 07:16:05.785244 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.785223 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Nov 04 07:16:05.785244 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.785238 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Nov 04 07:16:05.785378 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.785282 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Nov 04 07:16:05.785378 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.785298 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Nov 04 07:16:05.785449 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.785429 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Nov 04 07:16:05.785520 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.785491 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Nov 04 07:16:05.785568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.785537 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Nov 04 07:16:05.785605 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.785595 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Nov 04 07:16:05.785792 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.785777 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 04 07:16:05.785831 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.785791 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Nov 04 07:16:05.785831 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.785797 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Nov 04 07:16:05.791310 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.791287 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Nov 04 07:16:05.791540 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.791500 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Nov 04 07:16:05.868190 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.868162 5385 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 07:16:05.878022 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.878000 5385 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.878164 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.878152 5385 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.886158 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886135 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-service-ca\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:16:05.886261 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886164 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqkrc\" (UniqueName: \"kubernetes.io/projected/f33f6c60-eff2-40c2-9350-ba10eaf5442d-kube-api-access-bqkrc\") pod \"node-ca-sd2bs\" (UID: \"f33f6c60-eff2-40c2-9350-ba10eaf5442d\") " pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:16:05.886261 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886184 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-modprobe-d\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.886261 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886211 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.886261 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886238 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-systemd-units\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.886468 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886326 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/51538327-34ef-409b-b76a-6963b7254eec-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:16:05.886468 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886402 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fa9b645b-69d5-47a7-8457-409364209023-kubelet-config\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:16:05.886468 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886440 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-oauth-serving-cert\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:16:05.886468 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886443 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-service-ca\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:16:05.886468 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886466 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-device-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.886716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886526 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fa9b645b-69d5-47a7-8457-409364209023-dbus\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:16:05.886716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886556 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1172079a-2931-4ecb-9196-68e6da14fd36-tmp\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:16:05.886716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886618 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1172079a-2931-4ecb-9196-68e6da14fd36-tmp\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:16:05.886716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886619 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/51538327-34ef-409b-b76a-6963b7254eec-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:16:05.886716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886648 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9-agent-certs\") pod \"konnectivity-agent-2s2xx\" (UID: \"ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9\") " pod="kube-system/konnectivity-agent-2s2xx" Nov 04 07:16:05.886716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886686 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-run-netns\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.886716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886697 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-oauth-serving-cert\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:16:05.886983 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886719 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.886983 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886778 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.886983 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886828 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhnsv\" (UniqueName: \"kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv\") pod \"network-check-target-9qgxt\" (UID: \"97707149-0248-45a0-87d2-7347b960c2db\") " pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:16:05.886983 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886862 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.886983 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886893 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/51538327-34ef-409b-b76a-6963b7254eec-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:16:05.886983 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886916 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9-agent-certs\") pod \"konnectivity-agent-2s2xx\" (UID: \"ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9\") " pod="kube-system/konnectivity-agent-2s2xx" Nov 04 07:16:05.886983 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.886952 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19b2ab56-a868-4570-b098-37c517da236a-tmp-dir\") pod \"node-resolver-x6v9g\" (UID: \"19b2ab56-a868-4570-b098-37c517da236a\") " pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:16:05.887292 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887000 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.887292 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887080 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-sys\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.887292 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887120 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19b2ab56-a868-4570-b098-37c517da236a-tmp-dir\") pod \"node-resolver-x6v9g\" (UID: \"19b2ab56-a868-4570-b098-37c517da236a\") " pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:16:05.887292 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887136 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/51538327-34ef-409b-b76a-6963b7254eec-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:16:05.887292 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887127 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.887292 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887183 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:16:05.887292 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887219 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.887292 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887255 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-serving-cert\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:05.887650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887309 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-service-ca\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:05.887650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887330 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klgh7\" (UniqueName: \"kubernetes.io/projected/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-kube-api-access-klgh7\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.887650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887345 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9-konnectivity-ca\") pod \"konnectivity-agent-2s2xx\" (UID: \"ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9\") " pod="kube-system/konnectivity-agent-2s2xx" Nov 04 07:16:05.887650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887359 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.887650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887383 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-oauth-config\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:05.887650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887395 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:16:05.887650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887412 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-k8s-cni-cncf-io\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.887650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887462 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-serving-cert\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:05.887650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887464 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-conf-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.887650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887555 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysctl-d\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.887650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887594 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-lib-modules\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.887650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887597 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-service-ca\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:05.887650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887635 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.887650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887644 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887677 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-oauth-config\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887687 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jmmp\" (UniqueName: \"kubernetes.io/projected/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-kube-api-access-5jmmp\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887720 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63631d09-f1c5-43a6-98e9-4002ada7949a-host-slash\") pod \"iptables-alerter-s8gfp\" (UID: \"63631d09-f1c5-43a6-98e9-4002ada7949a\") " pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887735 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-log-socket\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887750 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-kubernetes\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887766 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-var-lib-kubelet\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887788 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-cni-binary-copy\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887797 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887821 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f33f6c60-eff2-40c2-9350-ba10eaf5442d-serviceca\") pod \"node-ca-sd2bs\" (UID: \"f33f6c60-eff2-40c2-9350-ba10eaf5442d\") " pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887850 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2dxb\" (UniqueName: \"kubernetes.io/projected/19b2ab56-a868-4570-b098-37c517da236a-kube-api-access-f2dxb\") pod \"node-resolver-x6v9g\" (UID: \"19b2ab56-a868-4570-b098-37c517da236a\") " pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887878 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-textfile\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887944 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887959 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-textfile\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887978 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-cni-binary-copy\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.887976 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovn-node-metrics-cert\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888013 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsljv\" (UniqueName: \"kubernetes.io/projected/17916760-be6c-47bf-8569-94ec0e36e304-kube-api-access-lsljv\") pod \"network-metrics-daemon-ldwvk\" (UID: \"17916760-be6c-47bf-8569-94ec0e36e304\") " pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888089 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t49f8\" (UniqueName: \"kubernetes.io/projected/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-kube-api-access-t49f8\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888117 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1488becb-97bc-43e6-b85c-6c9c62dfe903-metrics-client-ca\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888144 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888185 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888232 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovn-node-metrics-cert\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.888256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888235 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888274 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-multus-certs\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888305 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-node-log\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888332 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/63631d09-f1c5-43a6-98e9-4002ada7949a-iptables-alerter-script\") pod \"iptables-alerter-s8gfp\" (UID: \"63631d09-f1c5-43a6-98e9-4002ada7949a\") " pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888354 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f33f6c60-eff2-40c2-9350-ba10eaf5442d-serviceca\") pod \"node-ca-sd2bs\" (UID: \"f33f6c60-eff2-40c2-9350-ba10eaf5442d\") " pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888380 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888404 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-run-ovn-kubernetes\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888430 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-systemd\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888454 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-wtmp\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888493 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-cni-binary-copy\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888539 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-registration-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888734 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-cni-binary-copy\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888750 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/63631d09-f1c5-43a6-98e9-4002ada7949a-iptables-alerter-script\") pod \"iptables-alerter-s8gfp\" (UID: \"63631d09-f1c5-43a6-98e9-4002ada7949a\") " pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888842 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-tuned\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888877 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-tmp\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888908 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-web-config\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888915 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.888981 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1488becb-97bc-43e6-b85c-6c9c62dfe903-metrics-client-ca\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889007 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-tmp\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889027 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889022 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-tuning-conf-dir\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889066 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-tuned\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889099 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgllt\" (UniqueName: \"kubernetes.io/projected/63631d09-f1c5-43a6-98e9-4002ada7949a-kube-api-access-mgllt\") pod \"iptables-alerter-s8gfp\" (UID: \"63631d09-f1c5-43a6-98e9-4002ada7949a\") " pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889121 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-oauth-config\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889148 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-sys-fs\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889215 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-etc-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889265 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-accelerators-collector-config\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889293 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9-konnectivity-ca\") pod \"konnectivity-agent-2s2xx\" (UID: \"ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9\") " pod="kube-system/konnectivity-agent-2s2xx" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889306 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-cnibin\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:16:05.889329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889332 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d085676f-b286-4834-9e62-039f2b6e7f72-cert\") pod \"ingress-canary-pksw6\" (UID: \"d085676f-b286-4834-9e62-039f2b6e7f72\") " pod="openshift-ingress-canary/ingress-canary-pksw6" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889296 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-web-config\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889463 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-oauth-serving-cert\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889497 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-etc-kubernetes\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889529 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-oauth-config\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889583 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-env-overrides\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889612 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovnkube-script-lib\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889639 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f33f6c60-eff2-40c2-9350-ba10eaf5442d-host\") pod \"node-ca-sd2bs\" (UID: \"f33f6c60-eff2-40c2-9350-ba10eaf5442d\") " pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889666 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysctl-conf\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889695 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/508fee92-08e3-49ab-8b16-3add538b11eb-config-volume\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889699 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d085676f-b286-4834-9e62-039f2b6e7f72-cert\") pod \"ingress-canary-pksw6\" (UID: \"d085676f-b286-4834-9e62-039f2b6e7f72\") " pod="openshift-ingress-canary/ingress-canary-pksw6" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889699 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-accelerators-collector-config\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889746 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-sys\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889775 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-cnibin\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889821 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889865 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-env-overrides\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889877 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-host\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889900 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-oauth-serving-cert\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889910 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-config-volume\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889937 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-config-out\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.889965 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-crio-socket\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890068 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/508fee92-08e3-49ab-8b16-3add538b11eb-config-volume\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890088 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-console-config\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890090 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovnkube-script-lib\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890117 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-kubelet\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890145 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-systemd\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890188 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51538327-34ef-409b-b76a-6963b7254eec-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890217 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890263 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-config-volume\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890267 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-netns\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890278 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-console-config\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890295 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-cni-netd\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890342 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-config-out\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890449 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/80a4a0ad-5738-454b-9818-16b544aa5284-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:16:05.890603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890600 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51538327-34ef-409b-b76a-6963b7254eec-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890655 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2wdv\" (UniqueName: \"kubernetes.io/projected/d085676f-b286-4834-9e62-039f2b6e7f72-kube-api-access-x2wdv\") pod \"ingress-canary-pksw6\" (UID: \"d085676f-b286-4834-9e62-039f2b6e7f72\") " pod="openshift-ingress-canary/ingress-canary-pksw6" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890694 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-hostroot\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890767 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-daemon-config\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890836 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19b2ab56-a868-4570-b098-37c517da236a-hosts-file\") pod \"node-resolver-x6v9g\" (UID: \"19b2ab56-a868-4570-b098-37c517da236a\") " pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890913 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlj2h\" (UniqueName: \"kubernetes.io/projected/1488becb-97bc-43e6-b85c-6c9c62dfe903-kube-api-access-wlj2h\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890943 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hskwv\" (UniqueName: \"kubernetes.io/projected/1172079a-2931-4ecb-9196-68e6da14fd36-kube-api-access-hskwv\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.890963 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-daemon-config\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891081 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4rg8\" (UniqueName: \"kubernetes.io/projected/508fee92-08e3-49ab-8b16-3add538b11eb-kube-api-access-j4rg8\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891148 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891232 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30df6026-1baf-4fca-8c94-cb092397a676-tmp\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891264 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-cni-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891295 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-socket-dir-parent\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891348 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-socket-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891366 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30df6026-1baf-4fca-8c94-cb092397a676-tmp\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891394 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-cni-bin\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891426 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m7f6\" (UniqueName: \"kubernetes.io/projected/51538327-34ef-409b-b76a-6963b7254eec-kube-api-access-7m7f6\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891450 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891469 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-slash\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891541 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-tls\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891608 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqrrw\" (UniqueName: \"kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-kube-api-access-cqrrw\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891652 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-cni-bin\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891706 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtzpm\" (UniqueName: \"kubernetes.io/projected/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-kube-api-access-rtzpm\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891776 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysconfig\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891784 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-tls\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891804 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-root\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.891836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891830 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-run\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891873 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-data-volume\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891890 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891909 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-cni-multus\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891924 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-console-config\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891941 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-data-volume\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891944 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.891989 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovnkube-config\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892111 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892115 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892142 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-console-config\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892153 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vnql\" (UniqueName: \"kubernetes.io/projected/80a4a0ad-5738-454b-9818-16b544aa5284-kube-api-access-6vnql\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892252 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-ovnkube-config\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892264 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs\") pod \"network-metrics-daemon-ldwvk\" (UID: \"17916760-be6c-47bf-8569-94ec0e36e304\") " pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892291 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-system-cni-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892300 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892308 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2b6n\" (UniqueName: \"kubernetes.io/projected/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-kube-api-access-b2b6n\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892325 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-var-lib-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892370 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/508fee92-08e3-49ab-8b16-3add538b11eb-tmp-dir\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892433 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-trusted-ca-bundle\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892452 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-ovn\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892468 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892490 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-kubelet\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892537 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17916760-be6c-47bf-8569-94ec0e36e304-metrics-certs\") pod \"network-metrics-daemon-ldwvk\" (UID: \"17916760-be6c-47bf-8569-94ec0e36e304\") " pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892549 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-serving-cert\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892567 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/508fee92-08e3-49ab-8b16-3add538b11eb-tmp-dir\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892592 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/508fee92-08e3-49ab-8b16-3add538b11eb-metrics-tls\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892623 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892656 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-system-cni-dir\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892690 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzs4d\" (UniqueName: \"kubernetes.io/projected/30df6026-1baf-4fca-8c94-cb092397a676-kube-api-access-tzs4d\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:05.892915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892814 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-serving-cert\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:16:05.894558 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892965 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/fa9b645b-69d5-47a7-8457-409364209023-original-pull-secret\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:16:05.894558 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.892982 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/508fee92-08e3-49ab-8b16-3add538b11eb-metrics-tls\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:16:05.894558 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.893071 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.894558 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.893421 5385 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 07:16:05.894558 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.893465 5385 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 07:16:05.894558 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:05.893471 5385 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.894558 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:05.893521 5385 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-6.us-east-2.compute.internal" Nov 04 07:16:05.894558 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.894455 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-trusted-ca-bundle\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:05.901644 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.901628 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqkrc\" (UniqueName: \"kubernetes.io/projected/f33f6c60-eff2-40c2-9350-ba10eaf5442d-kube-api-access-bqkrc\") pod \"node-ca-sd2bs\" (UID: \"f33f6c60-eff2-40c2-9350-ba10eaf5442d\") " pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:16:05.905644 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.905618 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsljv\" (UniqueName: \"kubernetes.io/projected/17916760-be6c-47bf-8569-94ec0e36e304-kube-api-access-lsljv\") pod \"network-metrics-daemon-ldwvk\" (UID: \"17916760-be6c-47bf-8569-94ec0e36e304\") " pod="openshift-multus/network-metrics-daemon-ldwvk" Nov 04 07:16:05.905853 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.905836 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4rg8\" (UniqueName: \"kubernetes.io/projected/508fee92-08e3-49ab-8b16-3add538b11eb-kube-api-access-j4rg8\") pod \"dns-default-qpt88\" (UID: \"508fee92-08e3-49ab-8b16-3add538b11eb\") " pod="openshift-dns/dns-default-qpt88" Nov 04 07:16:05.909268 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.909240 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgllt\" (UniqueName: \"kubernetes.io/projected/63631d09-f1c5-43a6-98e9-4002ada7949a-kube-api-access-mgllt\") pod \"iptables-alerter-s8gfp\" (UID: \"63631d09-f1c5-43a6-98e9-4002ada7949a\") " pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:16:05.909268 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.909266 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2dxb\" (UniqueName: \"kubernetes.io/projected/19b2ab56-a868-4570-b098-37c517da236a-kube-api-access-f2dxb\") pod \"node-resolver-x6v9g\" (UID: \"19b2ab56-a868-4570-b098-37c517da236a\") " pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:16:05.909394 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.909374 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2b6n\" (UniqueName: \"kubernetes.io/projected/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-kube-api-access-b2b6n\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.910977 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.910951 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klgh7\" (UniqueName: \"kubernetes.io/projected/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-kube-api-access-klgh7\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.911123 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.911091 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqrrw\" (UniqueName: \"kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-kube-api-access-cqrrw\") pod \"alertmanager-main-0\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:05.911266 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.911248 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hskwv\" (UniqueName: \"kubernetes.io/projected/1172079a-2931-4ecb-9196-68e6da14fd36-kube-api-access-hskwv\") pod \"console-8656d77c48-qkjzp\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:16:05.911705 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.911682 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzs4d\" (UniqueName: \"kubernetes.io/projected/30df6026-1baf-4fca-8c94-cb092397a676-kube-api-access-tzs4d\") pod \"console-d75fb8d4b-tj7rv\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:05.911831 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.911811 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhnsv\" (UniqueName: \"kubernetes.io/projected/97707149-0248-45a0-87d2-7347b960c2db-kube-api-access-nhnsv\") pod \"network-check-target-9qgxt\" (UID: \"97707149-0248-45a0-87d2-7347b960c2db\") " pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:16:05.913356 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.913335 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlj2h\" (UniqueName: \"kubernetes.io/projected/1488becb-97bc-43e6-b85c-6c9c62dfe903-kube-api-access-wlj2h\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.914494 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.914475 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m7f6\" (UniqueName: \"kubernetes.io/projected/51538327-34ef-409b-b76a-6963b7254eec-kube-api-access-7m7f6\") pod \"prometheus-operator-9cd89b74c-sh7s5\" (UID: \"51538327-34ef-409b-b76a-6963b7254eec\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-sh7s5" Nov 04 07:16:05.914609 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.914592 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtzpm\" (UniqueName: \"kubernetes.io/projected/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-kube-api-access-rtzpm\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.915236 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.915220 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jmmp\" (UniqueName: \"kubernetes.io/projected/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-kube-api-access-5jmmp\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:16:05.915790 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.915774 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vnql\" (UniqueName: \"kubernetes.io/projected/80a4a0ad-5738-454b-9818-16b544aa5284-kube-api-access-6vnql\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:16:05.916000 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.915983 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t49f8\" (UniqueName: \"kubernetes.io/projected/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-kube-api-access-t49f8\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.917497 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.917479 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2wdv\" (UniqueName: \"kubernetes.io/projected/d085676f-b286-4834-9e62-039f2b6e7f72-kube-api-access-x2wdv\") pod \"ingress-canary-pksw6\" (UID: \"d085676f-b286-4834-9e62-039f2b6e7f72\") " pod="openshift-ingress-canary/ingress-canary-pksw6" Nov 04 07:16:05.965361 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.965294 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8656d77c48-qkjzp"] Nov 04 07:16:05.993686 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.993658 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f33f6c60-eff2-40c2-9350-ba10eaf5442d-host\") pod \"node-ca-sd2bs\" (UID: \"f33f6c60-eff2-40c2-9350-ba10eaf5442d\") " pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:16:05.993821 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.993691 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysctl-conf\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.993821 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.993709 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-sys\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.993821 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.993725 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-cnibin\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.993821 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.993746 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.993821 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.993793 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f33f6c60-eff2-40c2-9350-ba10eaf5442d-host\") pod \"node-ca-sd2bs\" (UID: \"f33f6c60-eff2-40c2-9350-ba10eaf5442d\") " pod="openshift-image-registry/node-ca-sd2bs" Nov 04 07:16:05.994007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.993827 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-host\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.994007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.993833 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-cnibin\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.994007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.993804 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-sys\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.994007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.993867 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.994007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.993862 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysctl-conf\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.994007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.993888 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-host\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.994007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.993922 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-crio-socket\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:16:05.994007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.993952 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-kubelet\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.994007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.993976 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-systemd\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.994007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994001 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-netns\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994007 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-kubelet\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994023 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-cni-netd\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994045 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-systemd\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994012 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7dcd4bbc-400c-4fce-b87c-0c018fadecfa-crio-socket\") pod \"insights-runtime-extractor-g6lhj\" (UID: \"7dcd4bbc-400c-4fce-b87c-0c018fadecfa\") " pod="openshift-insights/insights-runtime-extractor-g6lhj" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994056 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-netns\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994047 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-hostroot\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994080 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-hostroot\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994085 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19b2ab56-a868-4570-b098-37c517da236a-hosts-file\") pod \"node-resolver-x6v9g\" (UID: \"19b2ab56-a868-4570-b098-37c517da236a\") " pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994095 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-cni-netd\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994112 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/19b2ab56-a868-4570-b098-37c517da236a-hosts-file\") pod \"node-resolver-x6v9g\" (UID: \"19b2ab56-a868-4570-b098-37c517da236a\") " pod="openshift-dns/node-resolver-x6v9g" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994116 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-cni-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994138 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-socket-dir-parent\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994154 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-socket-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994167 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-cni-bin\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994210 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-slash\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994219 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-socket-dir-parent\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994226 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-socket-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994228 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-cni-bin\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994234 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-cni-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994249 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-slash\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994263 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-cni-bin\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994262 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-cni-bin\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.994307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994300 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysconfig\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994333 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-root\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994349 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysconfig\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994382 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-run\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994424 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-root\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994436 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-cni-multus\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994436 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-run\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994472 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994522 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-system-cni-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994533 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-cni-multus\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994548 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-var-lib-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994573 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-var-lib-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994573 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-system-cni-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994606 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-ovn\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994574 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-run-ovn\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994629 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-kubelet\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994549 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-etc-selinux\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994646 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-system-cni-dir\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994662 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-modprobe-d\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994677 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-systemd-units\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994681 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-system-cni-dir\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994663 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-var-lib-kubelet\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994691 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fa9b645b-69d5-47a7-8457-409364209023-kubelet-config\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994707 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-device-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994715 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-modprobe-d\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994721 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fa9b645b-69d5-47a7-8457-409364209023-dbus\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994739 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-run-netns\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994745 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-systemd-units\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994759 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-device-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994763 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/fa9b645b-69d5-47a7-8457-409364209023-kubelet-config\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994767 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994782 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/fa9b645b-69d5-47a7-8457-409364209023-dbus\") pod \"global-pull-secret-syncer-m7gt8\" (UID: \"fa9b645b-69d5-47a7-8457-409364209023\") " pod="kube-system/global-pull-secret-syncer-m7gt8" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994788 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-run-netns\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994802 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-sys\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994833 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-k8s-cni-cncf-io\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994856 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-sys\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994855 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-conf-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994880 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysctl-d\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994884 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-k8s-cni-cncf-io\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994894 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-lib-modules\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994802 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994894 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-multus-conf-dir\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994925 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63631d09-f1c5-43a6-98e9-4002ada7949a-host-slash\") pod \"iptables-alerter-s8gfp\" (UID: \"63631d09-f1c5-43a6-98e9-4002ada7949a\") " pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994941 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-log-socket\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994966 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-kubernetes\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994975 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-sysctl-d\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994979 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-var-lib-kubelet\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.994991 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-lib-modules\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995011 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-kubernetes\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995016 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-var-lib-kubelet\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995022 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/63631d09-f1c5-43a6-98e9-4002ada7949a-host-slash\") pod \"iptables-alerter-s8gfp\" (UID: \"63631d09-f1c5-43a6-98e9-4002ada7949a\") " pod="openshift-network-operator/iptables-alerter-s8gfp" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995038 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-log-socket\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995053 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-multus-certs\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995077 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-node-log\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995098 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-host-run-multus-certs\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995103 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995132 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-run-ovn-kubernetes\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995147 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995146 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-node-log\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995169 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-host-run-ovn-kubernetes\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995159 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-systemd\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995206 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-wtmp\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995227 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-registration-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995248 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-tuning-conf-dir\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995273 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-sys-fs\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995283 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1488becb-97bc-43e6-b85c-6c9c62dfe903-node-exporter-wtmp\") pod \"node-exporter-ktbsj\" (UID: \"1488becb-97bc-43e6-b85c-6c9c62dfe903\") " pod="openshift-monitoring/node-exporter-ktbsj" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995208 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6a43577d-3b09-4f31-89ec-5b4b7f7efb4c-etc-systemd\") pod \"tuned-dnvwf\" (UID: \"6a43577d-3b09-4f31-89ec-5b4b7f7efb4c\") " pod="openshift-cluster-node-tuning-operator/tuned-dnvwf" Nov 04 07:16:05.995258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995298 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-etc-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.997180 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995319 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-registration-dir\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.997180 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995326 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/30ff434f-8151-47d7-b6e9-51d9a3d8d6ad-etc-openvswitch\") pod \"ovnkube-node-95dnd\" (UID: \"30ff434f-8151-47d7-b6e9-51d9a3d8d6ad\") " pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:05.997180 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995332 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-tuning-conf-dir\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:16:05.997180 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995328 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-cnibin\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:16:05.997180 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995343 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fa408a9f-2280-4a77-bddd-c31dbe7fdb9c-sys-fs\") pod \"aws-ebs-csi-driver-node-xtfzl\" (UID: \"fa408a9f-2280-4a77-bddd-c31dbe7fdb9c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xtfzl" Nov 04 07:16:05.997180 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995364 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80a4a0ad-5738-454b-9818-16b544aa5284-cnibin\") pod \"multus-additional-cni-plugins-grc4v\" (UID: \"80a4a0ad-5738-454b-9818-16b544aa5284\") " pod="openshift-multus/multus-additional-cni-plugins-grc4v" Nov 04 07:16:05.997180 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995419 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-etc-kubernetes\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:05.997180 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:05.995453 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f-etc-kubernetes\") pod \"multus-6jbrj\" (UID: \"6d4ba0b2-5841-4bdc-88b1-4ee4b9c7db8f\") " pod="openshift-multus/multus-6jbrj" Nov 04 07:16:06.195401 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:06.195371 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:06.290959 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:06.290931 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:06.570161 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:06.570073 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:06.665086 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:06.665053 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:06.876324 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:06.876284 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:06.972991 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:06.972946 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:08.366769 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:08.366730 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qpt88" Nov 04 07:16:08.474650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:08.474620 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:08.480615 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:08.480591 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:08.562173 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:08.562149 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2s2xx" Nov 04 07:16:10.449875 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:10.449842 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:11.192834 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:11.192792 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:16:12.074919 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:12.074884 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:12.149104 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:12.149067 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:16:12.176834 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:12.176793 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:12.680888 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:12.680858 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:14.077319 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:14.077291 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:14.077740 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:14.077489 5385 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 07:16:14.774779 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:14.774744 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:14.980848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:14.980785 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:15.143068 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:15.143031 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:15.303749 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:15.303720 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:15.425616 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:15.425494 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:15.526694 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:15.526664 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:20.454446 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:20.454368 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:16:20.663123 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:20.663099 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:23.965945 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:23.965899 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:24.062777 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:24.062731 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:27.105355 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:27.105260 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:27.221012 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:27.220975 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:29.269938 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:29.269905 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-95dnd" Nov 04 07:16:30.873799 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:30.873766 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:16:31.900772 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:31.900733 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-8656d77c48-qkjzp" podUID="1172079a-2931-4ecb-9196-68e6da14fd36" containerName="console" containerID="cri-o://0a23c53d443309b3605893289bc1ca3cfff8d36518735ff04cef4ed499816242" gracePeriod=15 Nov 04 07:16:32.128531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.128497 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:16:32.278744 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.278663 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1172079a-2931-4ecb-9196-68e6da14fd36-tmp\" (UniqueName: \"kubernetes.io/empty-dir/1172079a-2931-4ecb-9196-68e6da14fd36-tmp\") pod \"1172079a-2931-4ecb-9196-68e6da14fd36\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " Nov 04 07:16:32.278744 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.278703 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-console-config\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-console-config\") pod \"1172079a-2931-4ecb-9196-68e6da14fd36\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " Nov 04 07:16:32.278744 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.278725 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-oauth-serving-cert\") pod \"1172079a-2931-4ecb-9196-68e6da14fd36\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " Nov 04 07:16:32.278744 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.278746 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-oauth-config\") pod \"1172079a-2931-4ecb-9196-68e6da14fd36\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " Nov 04 07:16:32.279058 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.278795 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-service-ca\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-service-ca\") pod \"1172079a-2931-4ecb-9196-68e6da14fd36\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " Nov 04 07:16:32.279058 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.278814 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-serving-cert\") pod \"1172079a-2931-4ecb-9196-68e6da14fd36\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " Nov 04 07:16:32.279058 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.278838 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1172079a-2931-4ecb-9196-68e6da14fd36-kube-api-access-hskwv\" (UniqueName: \"kubernetes.io/projected/1172079a-2931-4ecb-9196-68e6da14fd36-kube-api-access-hskwv\") pod \"1172079a-2931-4ecb-9196-68e6da14fd36\" (UID: \"1172079a-2931-4ecb-9196-68e6da14fd36\") " Nov 04 07:16:32.279058 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.278987 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1172079a-2931-4ecb-9196-68e6da14fd36-tmp" pod "1172079a-2931-4ecb-9196-68e6da14fd36" (UID: "1172079a-2931-4ecb-9196-68e6da14fd36"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:16:32.279268 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.279184 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-oauth-serving-cert" pod "1172079a-2931-4ecb-9196-68e6da14fd36" (UID: "1172079a-2931-4ecb-9196-68e6da14fd36"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:16:32.279268 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.279193 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-console-config" pod "1172079a-2931-4ecb-9196-68e6da14fd36" (UID: "1172079a-2931-4ecb-9196-68e6da14fd36"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:16:32.279365 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.279291 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-service-ca" pod "1172079a-2931-4ecb-9196-68e6da14fd36" (UID: "1172079a-2931-4ecb-9196-68e6da14fd36"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:16:32.281154 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.281121 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-oauth-config" pod "1172079a-2931-4ecb-9196-68e6da14fd36" (UID: "1172079a-2931-4ecb-9196-68e6da14fd36"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:16:32.281274 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.281225 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1172079a-2931-4ecb-9196-68e6da14fd36-kube-api-access-hskwv" pod "1172079a-2931-4ecb-9196-68e6da14fd36" (UID: "1172079a-2931-4ecb-9196-68e6da14fd36"). InnerVolumeSpecName "kube-api-access-hskwv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:16:32.281274 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.281240 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-serving-cert" pod "1172079a-2931-4ecb-9196-68e6da14fd36" (UID: "1172079a-2931-4ecb-9196-68e6da14fd36"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:16:32.353103 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.353020 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:32.381349 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.381314 5385 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1172079a-2931-4ecb-9196-68e6da14fd36-tmp\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:32.381349 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.381344 5385 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-console-config\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:32.381579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.381358 5385 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-oauth-serving-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:32.381579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.381371 5385 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-oauth-config\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:32.381579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.381385 5385 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1172079a-2931-4ecb-9196-68e6da14fd36-service-ca\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:32.381579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.381397 5385 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1172079a-2931-4ecb-9196-68e6da14fd36-console-serving-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:32.381579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.381409 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hskwv\" (UniqueName: \"kubernetes.io/projected/1172079a-2931-4ecb-9196-68e6da14fd36-kube-api-access-hskwv\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:32.663316 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.663283 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:32.964782 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.964699 5385 generic.go:356] "Generic (PLEG): container finished" podID="1172079a-2931-4ecb-9196-68e6da14fd36" containerID="0a23c53d443309b3605893289bc1ca3cfff8d36518735ff04cef4ed499816242" exitCode=0 Nov 04 07:16:32.964782 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.964753 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8656d77c48-qkjzp" Nov 04 07:16:32.964782 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.964771 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8656d77c48-qkjzp" event={"ID":"1172079a-2931-4ecb-9196-68e6da14fd36","Type":"ContainerDied","Data":"0a23c53d443309b3605893289bc1ca3cfff8d36518735ff04cef4ed499816242"} Nov 04 07:16:32.965246 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.964815 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8656d77c48-qkjzp" event={"ID":"1172079a-2931-4ecb-9196-68e6da14fd36","Type":"ContainerDied","Data":"fc16b866b67f1e6c70bb4ee91fdc2a6fb1b7120131aaaee82fb4f1b3a30bcc0d"} Nov 04 07:16:32.965246 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.964839 5385 scope.go:122] "RemoveContainer" containerID="0a23c53d443309b3605893289bc1ca3cfff8d36518735ff04cef4ed499816242" Nov 04 07:16:32.972687 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.972663 5385 scope.go:122] "RemoveContainer" containerID="0a23c53d443309b3605893289bc1ca3cfff8d36518735ff04cef4ed499816242" Nov 04 07:16:32.972960 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:32.972940 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a23c53d443309b3605893289bc1ca3cfff8d36518735ff04cef4ed499816242\": container with ID starting with 0a23c53d443309b3605893289bc1ca3cfff8d36518735ff04cef4ed499816242 not found: ID does not exist" containerID="0a23c53d443309b3605893289bc1ca3cfff8d36518735ff04cef4ed499816242" Nov 04 07:16:32.973029 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.972967 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a23c53d443309b3605893289bc1ca3cfff8d36518735ff04cef4ed499816242"} err="failed to get container status \"0a23c53d443309b3605893289bc1ca3cfff8d36518735ff04cef4ed499816242\": rpc error: code = NotFound desc = could not find container \"0a23c53d443309b3605893289bc1ca3cfff8d36518735ff04cef4ed499816242\": container with ID starting with 0a23c53d443309b3605893289bc1ca3cfff8d36518735ff04cef4ed499816242 not found: ID does not exist" Nov 04 07:16:32.981804 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.981782 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8656d77c48-qkjzp"] Nov 04 07:16:32.984657 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:32.984633 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8656d77c48-qkjzp"] Nov 04 07:16:33.408401 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:33.408363 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 07:16:33.408812 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:33.408786 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="alertmanager" containerID="cri-o://eb9f92288969284551c442cc77eca55e870bed0bda8dac28eca9da37d2117fdf" gracePeriod=120 Nov 04 07:16:33.408918 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:33.408856 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="kube-rbac-proxy-metric" containerID="cri-o://025944af5d89221999b3afc40050fb1b38c43b5676d7d9e2882312b4e7ec55ee" gracePeriod=120 Nov 04 07:16:33.408918 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:33.408884 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="prom-label-proxy" containerID="cri-o://0ed20dd35961d617c1408d79b8677652ee6d9262c30f2092fdddd6ad301df166" gracePeriod=120 Nov 04 07:16:33.409023 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:33.408897 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="kube-rbac-proxy-web" containerID="cri-o://3ce87f1d9b05063aadb063e169720dd31a27f37360fd7e6aa7aa6c96a7bb5ce0" gracePeriod=120 Nov 04 07:16:33.409023 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:33.408905 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="kube-rbac-proxy" containerID="cri-o://beee87a33b0b7ccbf013408ae24b9a234f241be68c6af640b7d3ed689d020802" gracePeriod=120 Nov 04 07:16:33.409023 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:33.408919 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="config-reloader" containerID="cri-o://0006278638bddf21a3ff6d49fc9d393a076eb64c9da8c81195994fc999b742d3" gracePeriod=120 Nov 04 07:16:33.971443 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:33.971407 5385 generic.go:356] "Generic (PLEG): container finished" podID="3632b027-6f63-4015-a775-b3a7d6d32247" containerID="0ed20dd35961d617c1408d79b8677652ee6d9262c30f2092fdddd6ad301df166" exitCode=0 Nov 04 07:16:33.971443 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:33.971435 5385 generic.go:356] "Generic (PLEG): container finished" podID="3632b027-6f63-4015-a775-b3a7d6d32247" containerID="025944af5d89221999b3afc40050fb1b38c43b5676d7d9e2882312b4e7ec55ee" exitCode=0 Nov 04 07:16:33.971443 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:33.971442 5385 generic.go:356] "Generic (PLEG): container finished" podID="3632b027-6f63-4015-a775-b3a7d6d32247" containerID="beee87a33b0b7ccbf013408ae24b9a234f241be68c6af640b7d3ed689d020802" exitCode=0 Nov 04 07:16:33.971443 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:33.971447 5385 generic.go:356] "Generic (PLEG): container finished" podID="3632b027-6f63-4015-a775-b3a7d6d32247" containerID="0006278638bddf21a3ff6d49fc9d393a076eb64c9da8c81195994fc999b742d3" exitCode=0 Nov 04 07:16:33.971974 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:33.971453 5385 generic.go:356] "Generic (PLEG): container finished" podID="3632b027-6f63-4015-a775-b3a7d6d32247" containerID="eb9f92288969284551c442cc77eca55e870bed0bda8dac28eca9da37d2117fdf" exitCode=0 Nov 04 07:16:33.971974 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:33.971481 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3632b027-6f63-4015-a775-b3a7d6d32247","Type":"ContainerDied","Data":"0ed20dd35961d617c1408d79b8677652ee6d9262c30f2092fdddd6ad301df166"} Nov 04 07:16:33.971974 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:33.971586 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3632b027-6f63-4015-a775-b3a7d6d32247","Type":"ContainerDied","Data":"025944af5d89221999b3afc40050fb1b38c43b5676d7d9e2882312b4e7ec55ee"} Nov 04 07:16:33.971974 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:33.971629 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3632b027-6f63-4015-a775-b3a7d6d32247","Type":"ContainerDied","Data":"beee87a33b0b7ccbf013408ae24b9a234f241be68c6af640b7d3ed689d020802"} Nov 04 07:16:33.971974 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:33.971655 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3632b027-6f63-4015-a775-b3a7d6d32247","Type":"ContainerDied","Data":"0006278638bddf21a3ff6d49fc9d393a076eb64c9da8c81195994fc999b742d3"} Nov 04 07:16:33.971974 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:33.971678 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3632b027-6f63-4015-a775-b3a7d6d32247","Type":"ContainerDied","Data":"eb9f92288969284551c442cc77eca55e870bed0bda8dac28eca9da37d2117fdf"} Nov 04 07:16:34.186746 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.186708 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:34.315619 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.315546 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:34.423758 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.423725 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:34.647707 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.647684 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:34.801265 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.801238 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-web\") pod \"3632b027-6f63-4015-a775-b3a7d6d32247\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " Nov 04 07:16:34.801458 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.801283 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy\") pod \"3632b027-6f63-4015-a775-b3a7d6d32247\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " Nov 04 07:16:34.801458 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.801318 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-main-tls\") pod \"3632b027-6f63-4015-a775-b3a7d6d32247\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " Nov 04 07:16:34.801458 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.801350 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-kube-api-access-cqrrw\" (UniqueName: \"kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-kube-api-access-cqrrw\") pod \"3632b027-6f63-4015-a775-b3a7d6d32247\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " Nov 04 07:16:34.801458 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.801393 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-tls-assets\" (UniqueName: \"kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-tls-assets\") pod \"3632b027-6f63-4015-a775-b3a7d6d32247\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " Nov 04 07:16:34.801458 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.801440 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-metric\") pod \"3632b027-6f63-4015-a775-b3a7d6d32247\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " Nov 04 07:16:34.801746 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.801465 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-config-out\" (UniqueName: \"kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-config-out\") pod \"3632b027-6f63-4015-a775-b3a7d6d32247\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " Nov 04 07:16:34.801746 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.801497 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-cluster-tls-config\") pod \"3632b027-6f63-4015-a775-b3a7d6d32247\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " Nov 04 07:16:34.801746 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.801547 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-config-volume\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-config-volume\") pod \"3632b027-6f63-4015-a775-b3a7d6d32247\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " Nov 04 07:16:34.801746 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.801579 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-metrics-client-ca\") pod \"3632b027-6f63-4015-a775-b3a7d6d32247\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " Nov 04 07:16:34.801746 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.801609 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-trusted-ca-bundle\") pod \"3632b027-6f63-4015-a775-b3a7d6d32247\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " Nov 04 07:16:34.801746 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.801647 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-main-db\") pod \"3632b027-6f63-4015-a775-b3a7d6d32247\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " Nov 04 07:16:34.801746 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.801691 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-web-config\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-web-config\") pod \"3632b027-6f63-4015-a775-b3a7d6d32247\" (UID: \"3632b027-6f63-4015-a775-b3a7d6d32247\") " Nov 04 07:16:34.804281 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.804209 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-main-db" pod "3632b027-6f63-4015-a775-b3a7d6d32247" (UID: "3632b027-6f63-4015-a775-b3a7d6d32247"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:16:34.804281 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.804229 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-trusted-ca-bundle" pod "3632b027-6f63-4015-a775-b3a7d6d32247" (UID: "3632b027-6f63-4015-a775-b3a7d6d32247"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:16:34.804281 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.804219 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-metrics-client-ca" pod "3632b027-6f63-4015-a775-b3a7d6d32247" (UID: "3632b027-6f63-4015-a775-b3a7d6d32247"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:16:34.804943 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.804915 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-tls-assets" pod "3632b027-6f63-4015-a775-b3a7d6d32247" (UID: "3632b027-6f63-4015-a775-b3a7d6d32247"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:16:34.805046 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.804917 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy" pod "3632b027-6f63-4015-a775-b3a7d6d32247" (UID: "3632b027-6f63-4015-a775-b3a7d6d32247"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:16:34.805552 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.805496 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-main-tls" pod "3632b027-6f63-4015-a775-b3a7d6d32247" (UID: "3632b027-6f63-4015-a775-b3a7d6d32247"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:16:34.805675 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.805452 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-web" pod "3632b027-6f63-4015-a775-b3a7d6d32247" (UID: "3632b027-6f63-4015-a775-b3a7d6d32247"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:16:34.805806 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.805783 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-kube-api-access-cqrrw" pod "3632b027-6f63-4015-a775-b3a7d6d32247" (UID: "3632b027-6f63-4015-a775-b3a7d6d32247"). InnerVolumeSpecName "kube-api-access-cqrrw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:16:34.805872 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.805813 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-config-out" pod "3632b027-6f63-4015-a775-b3a7d6d32247" (UID: "3632b027-6f63-4015-a775-b3a7d6d32247"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:16:34.805929 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.805885 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-metric" pod "3632b027-6f63-4015-a775-b3a7d6d32247" (UID: "3632b027-6f63-4015-a775-b3a7d6d32247"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:16:34.806320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.806292 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-config-volume" pod "3632b027-6f63-4015-a775-b3a7d6d32247" (UID: "3632b027-6f63-4015-a775-b3a7d6d32247"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:16:34.810264 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.810230 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-cluster-tls-config" pod "3632b027-6f63-4015-a775-b3a7d6d32247" (UID: "3632b027-6f63-4015-a775-b3a7d6d32247"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:16:34.811393 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.811372 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1172079a-2931-4ecb-9196-68e6da14fd36" path="/var/lib/kubelet/pods/1172079a-2931-4ecb-9196-68e6da14fd36/volumes" Nov 04 07:16:34.815994 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.815975 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-web-config" pod "3632b027-6f63-4015-a775-b3a7d6d32247" (UID: "3632b027-6f63-4015-a775-b3a7d6d32247"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:16:34.902150 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.902120 5385 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:34.902150 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.902150 5385 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-config-out\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:34.902320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.902161 5385 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-cluster-tls-config\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:34.902320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.902170 5385 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-config-volume\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:34.902320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.902181 5385 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-metrics-client-ca\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:34.902320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.902191 5385 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:34.902320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.902200 5385 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3632b027-6f63-4015-a775-b3a7d6d32247-alertmanager-main-db\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:34.902320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.902230 5385 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-web-config\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:34.902320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.902240 5385 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:34.902320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.902249 5385 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:34.902320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.902257 5385 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3632b027-6f63-4015-a775-b3a7d6d32247-secret-alertmanager-main-tls\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:34.902320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.902265 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqrrw\" (UniqueName: \"kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-kube-api-access-cqrrw\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:34.902320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.902273 5385 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3632b027-6f63-4015-a775-b3a7d6d32247-tls-assets\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:16:34.978273 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.978241 5385 generic.go:356] "Generic (PLEG): container finished" podID="3632b027-6f63-4015-a775-b3a7d6d32247" containerID="3ce87f1d9b05063aadb063e169720dd31a27f37360fd7e6aa7aa6c96a7bb5ce0" exitCode=0 Nov 04 07:16:34.978786 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.978317 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3632b027-6f63-4015-a775-b3a7d6d32247","Type":"ContainerDied","Data":"3ce87f1d9b05063aadb063e169720dd31a27f37360fd7e6aa7aa6c96a7bb5ce0"} Nov 04 07:16:34.978786 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.978335 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:34.978786 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.978386 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3632b027-6f63-4015-a775-b3a7d6d32247","Type":"ContainerDied","Data":"2a0c856154e6e0535c5bc1b0fc56fd5269d08bac276190c963ee0aa3f660c6f4"} Nov 04 07:16:34.978786 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.978418 5385 scope.go:122] "RemoveContainer" containerID="0ed20dd35961d617c1408d79b8677652ee6d9262c30f2092fdddd6ad301df166" Nov 04 07:16:34.986271 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.986248 5385 scope.go:122] "RemoveContainer" containerID="025944af5d89221999b3afc40050fb1b38c43b5676d7d9e2882312b4e7ec55ee" Nov 04 07:16:34.992729 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.992711 5385 scope.go:122] "RemoveContainer" containerID="beee87a33b0b7ccbf013408ae24b9a234f241be68c6af640b7d3ed689d020802" Nov 04 07:16:34.999117 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:34.999099 5385 scope.go:122] "RemoveContainer" containerID="3ce87f1d9b05063aadb063e169720dd31a27f37360fd7e6aa7aa6c96a7bb5ce0" Nov 04 07:16:35.002961 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.002930 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 07:16:35.005731 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.005714 5385 scope.go:122] "RemoveContainer" containerID="0006278638bddf21a3ff6d49fc9d393a076eb64c9da8c81195994fc999b742d3" Nov 04 07:16:35.009158 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.009140 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 07:16:35.012363 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.012346 5385 scope.go:122] "RemoveContainer" containerID="eb9f92288969284551c442cc77eca55e870bed0bda8dac28eca9da37d2117fdf" Nov 04 07:16:35.018846 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.018822 5385 scope.go:122] "RemoveContainer" containerID="5782cea40175ed4b81900890fb9483ae7a3eccec289a6f13796e32865641f9ef" Nov 04 07:16:35.025553 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.025533 5385 scope.go:122] "RemoveContainer" containerID="0ed20dd35961d617c1408d79b8677652ee6d9262c30f2092fdddd6ad301df166" Nov 04 07:16:35.025850 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:35.025830 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ed20dd35961d617c1408d79b8677652ee6d9262c30f2092fdddd6ad301df166\": container with ID starting with 0ed20dd35961d617c1408d79b8677652ee6d9262c30f2092fdddd6ad301df166 not found: ID does not exist" containerID="0ed20dd35961d617c1408d79b8677652ee6d9262c30f2092fdddd6ad301df166" Nov 04 07:16:35.025941 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.025863 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ed20dd35961d617c1408d79b8677652ee6d9262c30f2092fdddd6ad301df166"} err="failed to get container status \"0ed20dd35961d617c1408d79b8677652ee6d9262c30f2092fdddd6ad301df166\": rpc error: code = NotFound desc = could not find container \"0ed20dd35961d617c1408d79b8677652ee6d9262c30f2092fdddd6ad301df166\": container with ID starting with 0ed20dd35961d617c1408d79b8677652ee6d9262c30f2092fdddd6ad301df166 not found: ID does not exist" Nov 04 07:16:35.025941 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.025888 5385 scope.go:122] "RemoveContainer" containerID="025944af5d89221999b3afc40050fb1b38c43b5676d7d9e2882312b4e7ec55ee" Nov 04 07:16:35.026147 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:35.026132 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"025944af5d89221999b3afc40050fb1b38c43b5676d7d9e2882312b4e7ec55ee\": container with ID starting with 025944af5d89221999b3afc40050fb1b38c43b5676d7d9e2882312b4e7ec55ee not found: ID does not exist" containerID="025944af5d89221999b3afc40050fb1b38c43b5676d7d9e2882312b4e7ec55ee" Nov 04 07:16:35.026208 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.026156 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"025944af5d89221999b3afc40050fb1b38c43b5676d7d9e2882312b4e7ec55ee"} err="failed to get container status \"025944af5d89221999b3afc40050fb1b38c43b5676d7d9e2882312b4e7ec55ee\": rpc error: code = NotFound desc = could not find container \"025944af5d89221999b3afc40050fb1b38c43b5676d7d9e2882312b4e7ec55ee\": container with ID starting with 025944af5d89221999b3afc40050fb1b38c43b5676d7d9e2882312b4e7ec55ee not found: ID does not exist" Nov 04 07:16:35.026208 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.026177 5385 scope.go:122] "RemoveContainer" containerID="beee87a33b0b7ccbf013408ae24b9a234f241be68c6af640b7d3ed689d020802" Nov 04 07:16:35.026399 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:35.026381 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"beee87a33b0b7ccbf013408ae24b9a234f241be68c6af640b7d3ed689d020802\": container with ID starting with beee87a33b0b7ccbf013408ae24b9a234f241be68c6af640b7d3ed689d020802 not found: ID does not exist" containerID="beee87a33b0b7ccbf013408ae24b9a234f241be68c6af640b7d3ed689d020802" Nov 04 07:16:35.026454 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.026407 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"beee87a33b0b7ccbf013408ae24b9a234f241be68c6af640b7d3ed689d020802"} err="failed to get container status \"beee87a33b0b7ccbf013408ae24b9a234f241be68c6af640b7d3ed689d020802\": rpc error: code = NotFound desc = could not find container \"beee87a33b0b7ccbf013408ae24b9a234f241be68c6af640b7d3ed689d020802\": container with ID starting with beee87a33b0b7ccbf013408ae24b9a234f241be68c6af640b7d3ed689d020802 not found: ID does not exist" Nov 04 07:16:35.026454 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.026427 5385 scope.go:122] "RemoveContainer" containerID="3ce87f1d9b05063aadb063e169720dd31a27f37360fd7e6aa7aa6c96a7bb5ce0" Nov 04 07:16:35.026720 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:35.026700 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ce87f1d9b05063aadb063e169720dd31a27f37360fd7e6aa7aa6c96a7bb5ce0\": container with ID starting with 3ce87f1d9b05063aadb063e169720dd31a27f37360fd7e6aa7aa6c96a7bb5ce0 not found: ID does not exist" containerID="3ce87f1d9b05063aadb063e169720dd31a27f37360fd7e6aa7aa6c96a7bb5ce0" Nov 04 07:16:35.026760 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.026725 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ce87f1d9b05063aadb063e169720dd31a27f37360fd7e6aa7aa6c96a7bb5ce0"} err="failed to get container status \"3ce87f1d9b05063aadb063e169720dd31a27f37360fd7e6aa7aa6c96a7bb5ce0\": rpc error: code = NotFound desc = could not find container \"3ce87f1d9b05063aadb063e169720dd31a27f37360fd7e6aa7aa6c96a7bb5ce0\": container with ID starting with 3ce87f1d9b05063aadb063e169720dd31a27f37360fd7e6aa7aa6c96a7bb5ce0 not found: ID does not exist" Nov 04 07:16:35.026760 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.026744 5385 scope.go:122] "RemoveContainer" containerID="0006278638bddf21a3ff6d49fc9d393a076eb64c9da8c81195994fc999b742d3" Nov 04 07:16:35.026960 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:35.026942 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0006278638bddf21a3ff6d49fc9d393a076eb64c9da8c81195994fc999b742d3\": container with ID starting with 0006278638bddf21a3ff6d49fc9d393a076eb64c9da8c81195994fc999b742d3 not found: ID does not exist" containerID="0006278638bddf21a3ff6d49fc9d393a076eb64c9da8c81195994fc999b742d3" Nov 04 07:16:35.027020 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.026968 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0006278638bddf21a3ff6d49fc9d393a076eb64c9da8c81195994fc999b742d3"} err="failed to get container status \"0006278638bddf21a3ff6d49fc9d393a076eb64c9da8c81195994fc999b742d3\": rpc error: code = NotFound desc = could not find container \"0006278638bddf21a3ff6d49fc9d393a076eb64c9da8c81195994fc999b742d3\": container with ID starting with 0006278638bddf21a3ff6d49fc9d393a076eb64c9da8c81195994fc999b742d3 not found: ID does not exist" Nov 04 07:16:35.027020 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.026987 5385 scope.go:122] "RemoveContainer" containerID="eb9f92288969284551c442cc77eca55e870bed0bda8dac28eca9da37d2117fdf" Nov 04 07:16:35.027246 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:35.027228 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb9f92288969284551c442cc77eca55e870bed0bda8dac28eca9da37d2117fdf\": container with ID starting with eb9f92288969284551c442cc77eca55e870bed0bda8dac28eca9da37d2117fdf not found: ID does not exist" containerID="eb9f92288969284551c442cc77eca55e870bed0bda8dac28eca9da37d2117fdf" Nov 04 07:16:35.027316 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.027251 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb9f92288969284551c442cc77eca55e870bed0bda8dac28eca9da37d2117fdf"} err="failed to get container status \"eb9f92288969284551c442cc77eca55e870bed0bda8dac28eca9da37d2117fdf\": rpc error: code = NotFound desc = could not find container \"eb9f92288969284551c442cc77eca55e870bed0bda8dac28eca9da37d2117fdf\": container with ID starting with eb9f92288969284551c442cc77eca55e870bed0bda8dac28eca9da37d2117fdf not found: ID does not exist" Nov 04 07:16:35.027316 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.027272 5385 scope.go:122] "RemoveContainer" containerID="5782cea40175ed4b81900890fb9483ae7a3eccec289a6f13796e32865641f9ef" Nov 04 07:16:35.027492 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:16:35.027477 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5782cea40175ed4b81900890fb9483ae7a3eccec289a6f13796e32865641f9ef\": container with ID starting with 5782cea40175ed4b81900890fb9483ae7a3eccec289a6f13796e32865641f9ef not found: ID does not exist" containerID="5782cea40175ed4b81900890fb9483ae7a3eccec289a6f13796e32865641f9ef" Nov 04 07:16:35.027552 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.027496 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5782cea40175ed4b81900890fb9483ae7a3eccec289a6f13796e32865641f9ef"} err="failed to get container status \"5782cea40175ed4b81900890fb9483ae7a3eccec289a6f13796e32865641f9ef\": rpc error: code = NotFound desc = could not find container \"5782cea40175ed4b81900890fb9483ae7a3eccec289a6f13796e32865641f9ef\": container with ID starting with 5782cea40175ed4b81900890fb9483ae7a3eccec289a6f13796e32865641f9ef not found: ID does not exist" Nov 04 07:16:35.042805 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.042783 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 07:16:35.042986 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.042974 5385 cpu_manager.go:414] "containerMap: removing container" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="kube-rbac-proxy-web" Nov 04 07:16:35.043051 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.042988 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="kube-rbac-proxy-web" Nov 04 07:16:35.043051 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.042999 5385 cpu_manager.go:414] "containerMap: removing container" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="init-config-reloader" Nov 04 07:16:35.043051 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043005 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="init-config-reloader" Nov 04 07:16:35.043051 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043015 5385 cpu_manager.go:414] "containerMap: removing container" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="kube-rbac-proxy-metric" Nov 04 07:16:35.043051 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043020 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="kube-rbac-proxy-metric" Nov 04 07:16:35.043051 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043026 5385 cpu_manager.go:414] "containerMap: removing container" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="config-reloader" Nov 04 07:16:35.043051 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043031 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="config-reloader" Nov 04 07:16:35.043051 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043044 5385 cpu_manager.go:414] "containerMap: removing container" podUID="1172079a-2931-4ecb-9196-68e6da14fd36" containerName="console" Nov 04 07:16:35.043051 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043048 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1172079a-2931-4ecb-9196-68e6da14fd36" containerName="console" Nov 04 07:16:35.043051 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043055 5385 cpu_manager.go:414] "containerMap: removing container" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="prom-label-proxy" Nov 04 07:16:35.043353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043060 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="prom-label-proxy" Nov 04 07:16:35.043353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043067 5385 cpu_manager.go:414] "containerMap: removing container" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="kube-rbac-proxy" Nov 04 07:16:35.043353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043071 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="kube-rbac-proxy" Nov 04 07:16:35.043353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043077 5385 cpu_manager.go:414] "containerMap: removing container" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="alertmanager" Nov 04 07:16:35.043353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043081 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="alertmanager" Nov 04 07:16:35.043353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043117 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="init-config-reloader" Nov 04 07:16:35.043353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043123 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="kube-rbac-proxy-metric" Nov 04 07:16:35.043353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043128 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="config-reloader" Nov 04 07:16:35.043353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043135 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="1172079a-2931-4ecb-9196-68e6da14fd36" containerName="console" Nov 04 07:16:35.043353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043140 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="prom-label-proxy" Nov 04 07:16:35.043353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043145 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="kube-rbac-proxy" Nov 04 07:16:35.043353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043151 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="alertmanager" Nov 04 07:16:35.043353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.043156 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" containerName="kube-rbac-proxy-web" Nov 04 07:16:35.046119 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.046104 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.051899 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.051881 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Nov 04 07:16:35.052007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.051915 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Nov 04 07:16:35.052007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.051926 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Nov 04 07:16:35.052007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.051932 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Nov 04 07:16:35.052161 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.052117 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Nov 04 07:16:35.052161 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.052131 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Nov 04 07:16:35.052161 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.052137 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Nov 04 07:16:35.052312 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.052227 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mgp2g\"" Nov 04 07:16:35.052364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.052348 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Nov 04 07:16:35.057702 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.057677 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Nov 04 07:16:35.066122 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.066102 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 07:16:35.204223 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.204139 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.204223 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.204179 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.204223 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.204215 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.204430 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.204311 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mz8l\" (UniqueName: \"kubernetes.io/projected/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-kube-api-access-8mz8l\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.204430 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.204347 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-web-config\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.204430 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.204373 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.204553 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.204444 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.204553 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.204476 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-config-out\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.204553 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.204494 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.204644 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.204568 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.204644 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.204596 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-config-volume\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.204702 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.204685 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.204734 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.204703 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.305712 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.305685 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mz8l\" (UniqueName: \"kubernetes.io/projected/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-kube-api-access-8mz8l\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.305821 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.305719 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-web-config\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.305821 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.305739 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.305821 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.305759 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.305821 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.305785 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-config-out\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.305821 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.305807 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.306020 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.305833 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.306020 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.305850 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-config-volume\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.306020 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.305870 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.306020 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.305896 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.306020 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.305927 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.306020 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.305960 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.306020 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.305992 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.306299 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.306247 5385 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 07:16:35.306494 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.306465 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.306823 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.306795 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.307456 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.307432 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.309229 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.309200 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.309318 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.309248 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-tls-assets\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.309318 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.309278 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-web-config\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.309318 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.309248 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-config-volume\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.309636 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.309608 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.309724 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.309657 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-config-out\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.309724 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.309704 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.309808 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.309788 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.310472 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.310450 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.318017 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.318000 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mz8l\" (UniqueName: \"kubernetes.io/projected/58fe8ba7-05e1-42f3-8354-aabdf391d0e4-kube-api-access-8mz8l\") pod \"alertmanager-main-0\" (UID: \"58fe8ba7-05e1-42f3-8354-aabdf391d0e4\") " pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.365995 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.365971 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Nov 04 07:16:35.647980 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.647948 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Nov 04 07:16:35.652347 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:35.652320 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58fe8ba7_05e1_42f3_8354_aabdf391d0e4.slice/crio-ff963b87681ab1aa965e3aa6a7e0ae4fe319c566871e6bbd13b793e2078c6f67 WatchSource:0}: Error finding container ff963b87681ab1aa965e3aa6a7e0ae4fe319c566871e6bbd13b793e2078c6f67: Status 404 returned error can't find the container with id ff963b87681ab1aa965e3aa6a7e0ae4fe319c566871e6bbd13b793e2078c6f67 Nov 04 07:16:35.982895 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.982861 5385 generic.go:356] "Generic (PLEG): container finished" podID="58fe8ba7-05e1-42f3-8354-aabdf391d0e4" containerID="571b44ea72c1db08b5c9961939c2d9c073c1ba7f43c1a9fcb978811ec0d7416e" exitCode=0 Nov 04 07:16:35.983307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.982945 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58fe8ba7-05e1-42f3-8354-aabdf391d0e4","Type":"ContainerDied","Data":"571b44ea72c1db08b5c9961939c2d9c073c1ba7f43c1a9fcb978811ec0d7416e"} Nov 04 07:16:35.983307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:35.983019 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58fe8ba7-05e1-42f3-8354-aabdf391d0e4","Type":"ContainerStarted","Data":"ff963b87681ab1aa965e3aa6a7e0ae4fe319c566871e6bbd13b793e2078c6f67"} Nov 04 07:16:36.810642 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:36.810610 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3632b027-6f63-4015-a775-b3a7d6d32247" path="/var/lib/kubelet/pods/3632b027-6f63-4015-a775-b3a7d6d32247/volumes" Nov 04 07:16:36.989916 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:36.989878 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58fe8ba7-05e1-42f3-8354-aabdf391d0e4","Type":"ContainerStarted","Data":"93d8a23713df7f991d53357eb00549baa8813c60769057efe483f6cc99b9047e"} Nov 04 07:16:36.990376 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:36.989957 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58fe8ba7-05e1-42f3-8354-aabdf391d0e4","Type":"ContainerStarted","Data":"058fcd6cf9595d0b4106b17275781127b91ead8fc956a602655337425166ebde"} Nov 04 07:16:36.990376 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:36.990003 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58fe8ba7-05e1-42f3-8354-aabdf391d0e4","Type":"ContainerStarted","Data":"7becdb464f75499a0063bc9478caeda4cb689729c00901be3c67e2b9284f9907"} Nov 04 07:16:36.990376 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:36.990042 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58fe8ba7-05e1-42f3-8354-aabdf391d0e4","Type":"ContainerStarted","Data":"41f8e187f62c0ec3921b96298f40f68254c22641343c3fb8c167815ff53c580c"} Nov 04 07:16:36.990376 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:36.990091 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58fe8ba7-05e1-42f3-8354-aabdf391d0e4","Type":"ContainerStarted","Data":"894f0facc586c4318417aa139a51f863374abcc95376b8adf667c80605863f67"} Nov 04 07:16:36.990376 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:36.990125 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"58fe8ba7-05e1-42f3-8354-aabdf391d0e4","Type":"ContainerStarted","Data":"216a106d22de6454996085fdffe44ce6b2d14cb3ec2fac6ae41cb4084c95c92c"} Nov 04 07:16:37.026468 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.026410 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.013928957 podStartE2EDuration="2.026396346s" totalImagesPullingTime="0s" totalInitContainerRuntime="12.467389ms" isStatefulPod=false podCreationTimestamp="2026-11-04 07:16:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:16:37.025751633 +0000 UTC m=+32.359691217" watchObservedRunningTime="2026-11-04 07:16:37.026396346 +0000 UTC m=+32.360335929" Nov 04 07:16:37.454422 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.454388 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6985d69b67-82vf8"] Nov 04 07:16:37.457741 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.457719 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.467942 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.467919 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Nov 04 07:16:37.471059 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.471037 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\"" Nov 04 07:16:37.472051 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.472024 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6985d69b67-82vf8"] Nov 04 07:16:37.529763 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.529729 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Nov 04 07:16:37.529917 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.529790 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-54rjm\"" Nov 04 07:16:37.529917 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.529795 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Nov 04 07:16:37.529917 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.529840 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Nov 04 07:16:37.529917 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.529881 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Nov 04 07:16:37.625141 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.625102 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/434549d4-e06f-4fb9-a903-9d53cb4646d5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.625141 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.625141 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/434549d4-e06f-4fb9-a903-9d53cb4646d5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.625310 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.625250 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-252ft\" (UniqueName: \"kubernetes.io/projected/434549d4-e06f-4fb9-a903-9d53cb4646d5-kube-api-access-252ft\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.625368 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.625310 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/434549d4-e06f-4fb9-a903-9d53cb4646d5-serving-certs-ca-bundle\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.625368 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.625341 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/434549d4-e06f-4fb9-a903-9d53cb4646d5-telemeter-client-tls\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.625368 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.625358 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/434549d4-e06f-4fb9-a903-9d53cb4646d5-secret-telemeter-client\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.625526 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.625403 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/434549d4-e06f-4fb9-a903-9d53cb4646d5-metrics-client-ca\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.625526 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.625473 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/434549d4-e06f-4fb9-a903-9d53cb4646d5-federate-client-tls\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.726861 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.726767 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-252ft\" (UniqueName: \"kubernetes.io/projected/434549d4-e06f-4fb9-a903-9d53cb4646d5-kube-api-access-252ft\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.726861 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.726808 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/434549d4-e06f-4fb9-a903-9d53cb4646d5-serving-certs-ca-bundle\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.726861 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.726837 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/434549d4-e06f-4fb9-a903-9d53cb4646d5-telemeter-client-tls\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.727146 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.726904 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/434549d4-e06f-4fb9-a903-9d53cb4646d5-secret-telemeter-client\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.727146 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.726945 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/434549d4-e06f-4fb9-a903-9d53cb4646d5-metrics-client-ca\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.727146 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.726974 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/434549d4-e06f-4fb9-a903-9d53cb4646d5-federate-client-tls\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.727146 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.727004 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/434549d4-e06f-4fb9-a903-9d53cb4646d5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.727146 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.727126 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/434549d4-e06f-4fb9-a903-9d53cb4646d5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.728030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.727707 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/434549d4-e06f-4fb9-a903-9d53cb4646d5-serving-certs-ca-bundle\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.728030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.727781 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/434549d4-e06f-4fb9-a903-9d53cb4646d5-metrics-client-ca\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.728030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.727896 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/434549d4-e06f-4fb9-a903-9d53cb4646d5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.730637 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.730614 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/434549d4-e06f-4fb9-a903-9d53cb4646d5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.730722 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.730697 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/434549d4-e06f-4fb9-a903-9d53cb4646d5-secret-telemeter-client\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.730816 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.730795 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/434549d4-e06f-4fb9-a903-9d53cb4646d5-telemeter-client-tls\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.730853 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.730819 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/434549d4-e06f-4fb9-a903-9d53cb4646d5-federate-client-tls\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.739773 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.739749 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-252ft\" (UniqueName: \"kubernetes.io/projected/434549d4-e06f-4fb9-a903-9d53cb4646d5-kube-api-access-252ft\") pod \"telemeter-client-6985d69b67-82vf8\" (UID: \"434549d4-e06f-4fb9-a903-9d53cb4646d5\") " pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:37.772464 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:37.772436 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" Nov 04 07:16:38.052983 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:38.052867 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6985d69b67-82vf8"] Nov 04 07:16:38.058436 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:38.058409 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod434549d4_e06f_4fb9_a903_9d53cb4646d5.slice/crio-c7133b8d665e0cec8538d01dcda316a721df2f066406c6efd877bc6755d3e48f WatchSource:0}: Error finding container c7133b8d665e0cec8538d01dcda316a721df2f066406c6efd877bc6755d3e48f: Status 404 returned error can't find the container with id c7133b8d665e0cec8538d01dcda316a721df2f066406c6efd877bc6755d3e48f Nov 04 07:16:38.060413 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:38.060395 5385 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 07:16:39.002098 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:39.002056 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" event={"ID":"434549d4-e06f-4fb9-a903-9d53cb4646d5","Type":"ContainerStarted","Data":"c7133b8d665e0cec8538d01dcda316a721df2f066406c6efd877bc6755d3e48f"} Nov 04 07:16:40.007612 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.007567 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" event={"ID":"434549d4-e06f-4fb9-a903-9d53cb4646d5","Type":"ContainerStarted","Data":"bdc43655d9140038110aaa1dbf776ceaf5aeb6c6708617c0ac431025cb2cf1f1"} Nov 04 07:16:40.008057 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.007673 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" event={"ID":"434549d4-e06f-4fb9-a903-9d53cb4646d5","Type":"ContainerStarted","Data":"f14041759a0905ac7f96789acb0a92240a4a879690107e9e08fcf751061f78e3"} Nov 04 07:16:40.008057 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.007704 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" event={"ID":"434549d4-e06f-4fb9-a903-9d53cb4646d5","Type":"ContainerStarted","Data":"4fd92937257f34572e8638a87601fcc8dbffc375898a62288850bdf4bf5c5a52"} Nov 04 07:16:40.036049 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.035999 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6985d69b67-82vf8" podStartSLOduration=1.486939675 podStartE2EDuration="3.035986537s" totalImagesPullingTime="1.549046862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:16:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:16:40.034240855 +0000 UTC m=+35.368180439" watchObservedRunningTime="2026-11-04 07:16:40.035986537 +0000 UTC m=+35.369926120" Nov 04 07:16:40.127731 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.127693 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-54f55cd7f-2b98k"] Nov 04 07:16:40.129746 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.129727 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.162102 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.162074 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54f55cd7f-2b98k"] Nov 04 07:16:40.250638 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.250602 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-oauth-config\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.250792 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.250655 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrrhn\" (UniqueName: \"kubernetes.io/projected/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-kube-api-access-wrrhn\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.250792 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.250735 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-config\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.250792 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.250770 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-service-ca\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.250901 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.250801 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-serving-cert\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.250901 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.250867 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-trusted-ca-bundle\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.250901 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.250887 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-oauth-serving-cert\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.250991 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.250967 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-tmp\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.256323 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.256296 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:40.352139 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.352076 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-trusted-ca-bundle\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.352139 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.352107 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-oauth-serving-cert\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.352139 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.352130 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-tmp\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.352365 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.352153 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-oauth-config\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.352365 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.352183 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrrhn\" (UniqueName: \"kubernetes.io/projected/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-kube-api-access-wrrhn\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.352365 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.352203 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-config\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.352365 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.352221 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-service-ca\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.352365 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.352251 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-serving-cert\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.352653 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.352582 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-tmp\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.352957 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.352927 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-oauth-serving-cert\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.353076 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.353011 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-service-ca\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.353076 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.353064 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-config\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.353240 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.353224 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-trusted-ca-bundle\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.355062 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.355038 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-oauth-config\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.355147 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.355047 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-serving-cert\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.358325 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.358302 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:40.364964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.364947 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrrhn\" (UniqueName: \"kubernetes.io/projected/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-kube-api-access-wrrhn\") pod \"console-54f55cd7f-2b98k\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.439934 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.439908 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:40.721623 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:40.721590 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-54f55cd7f-2b98k"] Nov 04 07:16:40.726601 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:16:40.726574 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b2b4f84_7be2_4fdc_a613_b7f84cabba8f.slice/crio-f34e3f87b585361360ef82790d02503011e1df3aa8f5c8119d2d6f69c946c308 WatchSource:0}: Error finding container f34e3f87b585361360ef82790d02503011e1df3aa8f5c8119d2d6f69c946c308: Status 404 returned error can't find the container with id f34e3f87b585361360ef82790d02503011e1df3aa8f5c8119d2d6f69c946c308 Nov 04 07:16:41.013122 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:41.013035 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f55cd7f-2b98k" event={"ID":"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f","Type":"ContainerStarted","Data":"246e0087a3694ca81b988a398b974c66390da014ed81f155b63545ac0fa244f1"} Nov 04 07:16:41.013122 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:41.013094 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f55cd7f-2b98k" event={"ID":"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f","Type":"ContainerStarted","Data":"f34e3f87b585361360ef82790d02503011e1df3aa8f5c8119d2d6f69c946c308"} Nov 04 07:16:41.034754 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:41.034710 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-54f55cd7f-2b98k" podStartSLOduration=1.0346950050000001 podStartE2EDuration="1.034695005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:16:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:16:41.033793945 +0000 UTC m=+36.367733529" watchObservedRunningTime="2026-11-04 07:16:41.034695005 +0000 UTC m=+36.368634588" Nov 04 07:16:41.347088 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:41.347010 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:41.446260 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:41.446228 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:42.075455 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:42.075416 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:42.151595 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:42.151567 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9qgxt" Nov 04 07:16:42.175952 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:42.175918 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:47.120910 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:47.120871 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:47.234144 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:47.234096 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:16:50.440315 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:50.440275 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:50.440706 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:50.440361 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:50.444652 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:50.444632 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:51.047561 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:51.047535 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:16:51.099860 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:16:51.099828 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d75fb8d4b-tj7rv"] Nov 04 07:17:00.873756 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:17:00.873717 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:17:07.697499 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:07.697463 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:17:07.792909 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:07.792878 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:17:07.980554 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:07.980463 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-78c87f586f-6twc6"] Nov 04 07:17:07.984830 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:07.984815 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:07.997287 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:07.997260 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78c87f586f-6twc6"] Nov 04 07:17:08.046165 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.046139 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ef794e0-f534-4536-b524-f40f82b32980-console-oauth-config\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.046282 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.046170 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ef794e0-f534-4536-b524-f40f82b32980-service-ca\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.046282 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.046190 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ef794e0-f534-4536-b524-f40f82b32980-trusted-ca-bundle\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.046282 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.046268 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ef794e0-f534-4536-b524-f40f82b32980-console-serving-cert\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.046409 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.046304 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ef794e0-f534-4536-b524-f40f82b32980-tmp\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.046409 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.046322 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ef794e0-f534-4536-b524-f40f82b32980-oauth-serving-cert\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.046409 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.046351 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t766n\" (UniqueName: \"kubernetes.io/projected/5ef794e0-f534-4536-b524-f40f82b32980-kube-api-access-t766n\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.046409 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.046369 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ef794e0-f534-4536-b524-f40f82b32980-console-config\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.063475 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.063451 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:17:08.147035 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.147010 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ef794e0-f534-4536-b524-f40f82b32980-service-ca\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.147134 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.147041 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ef794e0-f534-4536-b524-f40f82b32980-trusted-ca-bundle\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.147134 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.147070 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ef794e0-f534-4536-b524-f40f82b32980-console-serving-cert\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.147134 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.147088 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ef794e0-f534-4536-b524-f40f82b32980-tmp\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.147134 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.147104 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ef794e0-f534-4536-b524-f40f82b32980-oauth-serving-cert\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.147134 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.147126 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t766n\" (UniqueName: \"kubernetes.io/projected/5ef794e0-f534-4536-b524-f40f82b32980-kube-api-access-t766n\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.147370 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.147144 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ef794e0-f534-4536-b524-f40f82b32980-console-config\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.147370 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.147206 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ef794e0-f534-4536-b524-f40f82b32980-console-oauth-config\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.147565 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.147537 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ef794e0-f534-4536-b524-f40f82b32980-tmp\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.147992 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.147968 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5ef794e0-f534-4536-b524-f40f82b32980-oauth-serving-cert\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.148095 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.148034 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5ef794e0-f534-4536-b524-f40f82b32980-console-config\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.148095 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.148041 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5ef794e0-f534-4536-b524-f40f82b32980-service-ca\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.148208 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.148140 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ef794e0-f534-4536-b524-f40f82b32980-trusted-ca-bundle\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.149713 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.149693 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5ef794e0-f534-4536-b524-f40f82b32980-console-oauth-config\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.149713 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.149706 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ef794e0-f534-4536-b524-f40f82b32980-console-serving-cert\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.156802 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.156781 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:17:08.159263 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.159246 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t766n\" (UniqueName: \"kubernetes.io/projected/5ef794e0-f534-4536-b524-f40f82b32980-kube-api-access-t766n\") pod \"console-78c87f586f-6twc6\" (UID: \"5ef794e0-f534-4536-b524-f40f82b32980\") " pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.295738 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.295670 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:08.426549 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.426501 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:17:08.521068 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.521039 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:17:08.571017 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:08.570957 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78c87f586f-6twc6"] Nov 04 07:17:08.574116 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:17:08.574093 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef794e0_f534_4536_b524_f40f82b32980.slice/crio-8a2da169dcbbdf80380b528637c313fb786ab0ac5519ba14c0b6f57308f3572c WatchSource:0}: Error finding container 8a2da169dcbbdf80380b528637c313fb786ab0ac5519ba14c0b6f57308f3572c: Status 404 returned error can't find the container with id 8a2da169dcbbdf80380b528637c313fb786ab0ac5519ba14c0b6f57308f3572c Nov 04 07:17:09.104219 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:09.104182 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c87f586f-6twc6" event={"ID":"5ef794e0-f534-4536-b524-f40f82b32980","Type":"ContainerStarted","Data":"c1c4e54a8521da8bac33666d7b2e7862c1c1a2fb14ffea6ca546df5598d12206"} Nov 04 07:17:09.104590 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:09.104239 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78c87f586f-6twc6" event={"ID":"5ef794e0-f534-4536-b524-f40f82b32980","Type":"ContainerStarted","Data":"8a2da169dcbbdf80380b528637c313fb786ab0ac5519ba14c0b6f57308f3572c"} Nov 04 07:17:09.130863 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:09.130811 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-78c87f586f-6twc6" podStartSLOduration=2.130797229 podStartE2EDuration="2.130797229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:17:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:17:09.130293671 +0000 UTC m=+64.464233256" watchObservedRunningTime="2026-11-04 07:17:09.130797229 +0000 UTC m=+64.464736813" Nov 04 07:17:15.135409 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:15.135379 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:17:16.119221 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.119169 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-d75fb8d4b-tj7rv" podUID="30df6026-1baf-4fca-8c94-cb092397a676" containerName="console" containerID="cri-o://eebe19ee00c2d25a80cd76c9c1da0715ef261bf00407d399ca0b8a1560f74648" gracePeriod=15 Nov 04 07:17:16.358773 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.358748 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:17:16.412414 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.412331 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-oauth-config\") pod \"30df6026-1baf-4fca-8c94-cb092397a676\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " Nov 04 07:17:16.412414 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.412384 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-console-config\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-console-config\") pod \"30df6026-1baf-4fca-8c94-cb092397a676\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " Nov 04 07:17:16.412658 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.412420 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-service-ca\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-service-ca\") pod \"30df6026-1baf-4fca-8c94-cb092397a676\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " Nov 04 07:17:16.412658 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.412457 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/30df6026-1baf-4fca-8c94-cb092397a676-kube-api-access-tzs4d\" (UniqueName: \"kubernetes.io/projected/30df6026-1baf-4fca-8c94-cb092397a676-kube-api-access-tzs4d\") pod \"30df6026-1baf-4fca-8c94-cb092397a676\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " Nov 04 07:17:16.412658 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.412480 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-trusted-ca-bundle\") pod \"30df6026-1baf-4fca-8c94-cb092397a676\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " Nov 04 07:17:16.412658 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.412497 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-serving-cert\") pod \"30df6026-1baf-4fca-8c94-cb092397a676\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " Nov 04 07:17:16.412658 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.412623 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-oauth-serving-cert\") pod \"30df6026-1baf-4fca-8c94-cb092397a676\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " Nov 04 07:17:16.412915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.412684 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/30df6026-1baf-4fca-8c94-cb092397a676-tmp\" (UniqueName: \"kubernetes.io/empty-dir/30df6026-1baf-4fca-8c94-cb092397a676-tmp\") pod \"30df6026-1baf-4fca-8c94-cb092397a676\" (UID: \"30df6026-1baf-4fca-8c94-cb092397a676\") " Nov 04 07:17:16.412974 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.412919 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-service-ca" pod "30df6026-1baf-4fca-8c94-cb092397a676" (UID: "30df6026-1baf-4fca-8c94-cb092397a676"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:17:16.418194 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.412956 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-console-config" pod "30df6026-1baf-4fca-8c94-cb092397a676" (UID: "30df6026-1baf-4fca-8c94-cb092397a676"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:17:16.418194 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.413409 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-trusted-ca-bundle" pod "30df6026-1baf-4fca-8c94-cb092397a676" (UID: "30df6026-1baf-4fca-8c94-cb092397a676"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:17:16.418194 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.413760 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30df6026-1baf-4fca-8c94-cb092397a676-tmp" pod "30df6026-1baf-4fca-8c94-cb092397a676" (UID: "30df6026-1baf-4fca-8c94-cb092397a676"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:17:16.418194 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.413828 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-oauth-serving-cert" pod "30df6026-1baf-4fca-8c94-cb092397a676" (UID: "30df6026-1baf-4fca-8c94-cb092397a676"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:17:16.418194 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.415251 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30df6026-1baf-4fca-8c94-cb092397a676-kube-api-access-tzs4d" pod "30df6026-1baf-4fca-8c94-cb092397a676" (UID: "30df6026-1baf-4fca-8c94-cb092397a676"). InnerVolumeSpecName "kube-api-access-tzs4d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:17:16.418194 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.415270 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-serving-cert" pod "30df6026-1baf-4fca-8c94-cb092397a676" (UID: "30df6026-1baf-4fca-8c94-cb092397a676"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:17:16.418577 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.418342 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-oauth-config" pod "30df6026-1baf-4fca-8c94-cb092397a676" (UID: "30df6026-1baf-4fca-8c94-cb092397a676"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:17:16.513978 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.513945 5385 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-service-ca\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:17:16.513978 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.513974 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzs4d\" (UniqueName: \"kubernetes.io/projected/30df6026-1baf-4fca-8c94-cb092397a676-kube-api-access-tzs4d\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:17:16.514146 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.513992 5385 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-trusted-ca-bundle\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:17:16.514146 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.514002 5385 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-serving-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:17:16.514146 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.514012 5385 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-oauth-serving-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:17:16.514146 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.514020 5385 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30df6026-1baf-4fca-8c94-cb092397a676-tmp\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:17:16.514146 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.514029 5385 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/30df6026-1baf-4fca-8c94-cb092397a676-console-oauth-config\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:17:16.514146 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:16.514038 5385 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/30df6026-1baf-4fca-8c94-cb092397a676-console-config\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:17:17.133996 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:17.133966 5385 generic.go:356] "Generic (PLEG): container finished" podID="30df6026-1baf-4fca-8c94-cb092397a676" containerID="eebe19ee00c2d25a80cd76c9c1da0715ef261bf00407d399ca0b8a1560f74648" exitCode=0 Nov 04 07:17:17.134177 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:17.134030 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d75fb8d4b-tj7rv" Nov 04 07:17:17.134177 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:17.134053 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d75fb8d4b-tj7rv" event={"ID":"30df6026-1baf-4fca-8c94-cb092397a676","Type":"ContainerDied","Data":"eebe19ee00c2d25a80cd76c9c1da0715ef261bf00407d399ca0b8a1560f74648"} Nov 04 07:17:17.134177 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:17.134104 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d75fb8d4b-tj7rv" event={"ID":"30df6026-1baf-4fca-8c94-cb092397a676","Type":"ContainerDied","Data":"b158853477c105444b0009f84317f574595aad43b9046ecdb6eed47b76170255"} Nov 04 07:17:17.134177 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:17.134123 5385 scope.go:122] "RemoveContainer" containerID="eebe19ee00c2d25a80cd76c9c1da0715ef261bf00407d399ca0b8a1560f74648" Nov 04 07:17:17.141860 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:17.141840 5385 scope.go:122] "RemoveContainer" containerID="eebe19ee00c2d25a80cd76c9c1da0715ef261bf00407d399ca0b8a1560f74648" Nov 04 07:17:17.142148 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:17:17.142128 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eebe19ee00c2d25a80cd76c9c1da0715ef261bf00407d399ca0b8a1560f74648\": container with ID starting with eebe19ee00c2d25a80cd76c9c1da0715ef261bf00407d399ca0b8a1560f74648 not found: ID does not exist" containerID="eebe19ee00c2d25a80cd76c9c1da0715ef261bf00407d399ca0b8a1560f74648" Nov 04 07:17:17.142214 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:17.142157 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eebe19ee00c2d25a80cd76c9c1da0715ef261bf00407d399ca0b8a1560f74648"} err="failed to get container status \"eebe19ee00c2d25a80cd76c9c1da0715ef261bf00407d399ca0b8a1560f74648\": rpc error: code = NotFound desc = could not find container \"eebe19ee00c2d25a80cd76c9c1da0715ef261bf00407d399ca0b8a1560f74648\": container with ID starting with eebe19ee00c2d25a80cd76c9c1da0715ef261bf00407d399ca0b8a1560f74648 not found: ID does not exist" Nov 04 07:17:17.151691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:17.151669 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d75fb8d4b-tj7rv"] Nov 04 07:17:17.159435 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:17.159380 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d75fb8d4b-tj7rv"] Nov 04 07:17:18.296395 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:18.296355 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:18.296872 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:18.296431 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:18.300781 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:18.300759 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:18.809688 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:18.809655 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="30df6026-1baf-4fca-8c94-cb092397a676" path="/var/lib/kubelet/pods/30df6026-1baf-4fca-8c94-cb092397a676/volumes" Nov 04 07:17:19.144861 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:19.144834 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78c87f586f-6twc6" Nov 04 07:17:19.198755 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:19.198726 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54f55cd7f-2b98k"] Nov 04 07:17:21.422578 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:21.422537 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:17:21.518078 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:21.518046 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:17:27.087756 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:27.087679 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:17:27.192384 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:27.192358 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:17:30.872903 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:17:30.872863 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:17:38.663395 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:38.663364 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:17:42.627522 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:42.627480 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:17:44.217630 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.217575 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-54f55cd7f-2b98k" podUID="2b2b4f84-7be2-4fdc-a613-b7f84cabba8f" containerName="console" containerID="cri-o://246e0087a3694ca81b988a398b974c66390da014ed81f155b63545ac0fa244f1" gracePeriod=15 Nov 04 07:17:44.444347 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.444325 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:17:44.504481 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.504419 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-config\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-config\") pod \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " Nov 04 07:17:44.504481 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.504462 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-tmp\") pod \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " Nov 04 07:17:44.504662 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.504493 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-service-ca\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-service-ca\") pod \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " Nov 04 07:17:44.504662 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.504529 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-trusted-ca-bundle\") pod \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " Nov 04 07:17:44.504662 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.504563 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-oauth-serving-cert\") pod \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " Nov 04 07:17:44.504662 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.504585 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-oauth-config\") pod \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " Nov 04 07:17:44.504662 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.504640 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-kube-api-access-wrrhn\" (UniqueName: \"kubernetes.io/projected/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-kube-api-access-wrrhn\") pod \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " Nov 04 07:17:44.504922 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.504685 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-serving-cert\") pod \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\" (UID: \"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f\") " Nov 04 07:17:44.504922 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.504735 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-tmp" pod "2b2b4f84-7be2-4fdc-a613-b7f84cabba8f" (UID: "2b2b4f84-7be2-4fdc-a613-b7f84cabba8f"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:17:44.504922 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.504890 5385 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-tmp\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:17:44.504922 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.504915 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-config" pod "2b2b4f84-7be2-4fdc-a613-b7f84cabba8f" (UID: "2b2b4f84-7be2-4fdc-a613-b7f84cabba8f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:17:44.505127 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.504971 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-oauth-serving-cert" pod "2b2b4f84-7be2-4fdc-a613-b7f84cabba8f" (UID: "2b2b4f84-7be2-4fdc-a613-b7f84cabba8f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:17:44.505127 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.505013 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-trusted-ca-bundle" pod "2b2b4f84-7be2-4fdc-a613-b7f84cabba8f" (UID: "2b2b4f84-7be2-4fdc-a613-b7f84cabba8f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:17:44.505127 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.505026 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-service-ca" pod "2b2b4f84-7be2-4fdc-a613-b7f84cabba8f" (UID: "2b2b4f84-7be2-4fdc-a613-b7f84cabba8f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:17:44.506767 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.506740 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-serving-cert" pod "2b2b4f84-7be2-4fdc-a613-b7f84cabba8f" (UID: "2b2b4f84-7be2-4fdc-a613-b7f84cabba8f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:17:44.506869 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.506843 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-kube-api-access-wrrhn" pod "2b2b4f84-7be2-4fdc-a613-b7f84cabba8f" (UID: "2b2b4f84-7be2-4fdc-a613-b7f84cabba8f"). InnerVolumeSpecName "kube-api-access-wrrhn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:17:44.506923 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.506901 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-oauth-config" pod "2b2b4f84-7be2-4fdc-a613-b7f84cabba8f" (UID: "2b2b4f84-7be2-4fdc-a613-b7f84cabba8f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:17:44.605360 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.605335 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrrhn\" (UniqueName: \"kubernetes.io/projected/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-kube-api-access-wrrhn\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:17:44.605360 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.605356 5385 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-serving-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:17:44.605485 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.605367 5385 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-config\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:17:44.605485 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.605378 5385 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-service-ca\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:17:44.605485 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.605387 5385 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-trusted-ca-bundle\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:17:44.605485 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.605396 5385 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-oauth-serving-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:17:44.605485 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:44.605405 5385 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f-console-oauth-config\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:17:45.226090 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:45.226054 5385 generic.go:356] "Generic (PLEG): container finished" podID="2b2b4f84-7be2-4fdc-a613-b7f84cabba8f" containerID="246e0087a3694ca81b988a398b974c66390da014ed81f155b63545ac0fa244f1" exitCode=0 Nov 04 07:17:45.226584 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:45.226085 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f55cd7f-2b98k" event={"ID":"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f","Type":"ContainerDied","Data":"246e0087a3694ca81b988a398b974c66390da014ed81f155b63545ac0fa244f1"} Nov 04 07:17:45.226584 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:45.226114 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-54f55cd7f-2b98k" Nov 04 07:17:45.226584 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:45.226159 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-54f55cd7f-2b98k" event={"ID":"2b2b4f84-7be2-4fdc-a613-b7f84cabba8f","Type":"ContainerDied","Data":"f34e3f87b585361360ef82790d02503011e1df3aa8f5c8119d2d6f69c946c308"} Nov 04 07:17:45.226584 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:45.226179 5385 scope.go:122] "RemoveContainer" containerID="246e0087a3694ca81b988a398b974c66390da014ed81f155b63545ac0fa244f1" Nov 04 07:17:45.233790 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:45.233776 5385 scope.go:122] "RemoveContainer" containerID="246e0087a3694ca81b988a398b974c66390da014ed81f155b63545ac0fa244f1" Nov 04 07:17:45.234054 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:17:45.234035 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"246e0087a3694ca81b988a398b974c66390da014ed81f155b63545ac0fa244f1\": container with ID starting with 246e0087a3694ca81b988a398b974c66390da014ed81f155b63545ac0fa244f1 not found: ID does not exist" containerID="246e0087a3694ca81b988a398b974c66390da014ed81f155b63545ac0fa244f1" Nov 04 07:17:45.234117 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:45.234061 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"246e0087a3694ca81b988a398b974c66390da014ed81f155b63545ac0fa244f1"} err="failed to get container status \"246e0087a3694ca81b988a398b974c66390da014ed81f155b63545ac0fa244f1\": rpc error: code = NotFound desc = could not find container \"246e0087a3694ca81b988a398b974c66390da014ed81f155b63545ac0fa244f1\": container with ID starting with 246e0087a3694ca81b988a398b974c66390da014ed81f155b63545ac0fa244f1 not found: ID does not exist" Nov 04 07:17:45.243405 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:45.243383 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-54f55cd7f-2b98k"] Nov 04 07:17:45.247478 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:45.247457 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-54f55cd7f-2b98k"] Nov 04 07:17:46.809372 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:17:46.809303 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b2b4f84-7be2-4fdc-a613-b7f84cabba8f" path="/var/lib/kubelet/pods/2b2b4f84-7be2-4fdc-a613-b7f84cabba8f/volumes" Nov 04 07:18:00.872568 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:18:00.872525 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:18:15.192071 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:18:15.192031 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:18:28.484688 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:18:28.484651 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:18:28.584997 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:18:28.584961 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:18:30.873125 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:18:30.873082 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:19:00.872907 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:19:00.872870 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:19:01.184832 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:01.184733 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:15.244375 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:15.244338 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:21.662040 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:21.661937 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:21.786045 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:21.785986 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:21.899826 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:21.899794 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:22.890589 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.890551 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg"] Nov 04 07:19:22.890987 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.890939 5385 cpu_manager.go:414] "containerMap: removing container" podUID="2b2b4f84-7be2-4fdc-a613-b7f84cabba8f" containerName="console" Nov 04 07:19:22.890987 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.890957 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b2b4f84-7be2-4fdc-a613-b7f84cabba8f" containerName="console" Nov 04 07:19:22.890987 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.890972 5385 cpu_manager.go:414] "containerMap: removing container" podUID="30df6026-1baf-4fca-8c94-cb092397a676" containerName="console" Nov 04 07:19:22.890987 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.890980 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="30df6026-1baf-4fca-8c94-cb092397a676" containerName="console" Nov 04 07:19:22.891129 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.891087 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b2b4f84-7be2-4fdc-a613-b7f84cabba8f" containerName="console" Nov 04 07:19:22.891129 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.891099 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="30df6026-1baf-4fca-8c94-cb092397a676" containerName="console" Nov 04 07:19:22.894197 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.894177 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq"] Nov 04 07:19:22.894325 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.894311 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" Nov 04 07:19:22.897617 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.897601 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" Nov 04 07:19:22.902676 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.902656 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Nov 04 07:19:22.902765 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.902699 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-2m67d\"" Nov 04 07:19:22.902973 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.902963 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Nov 04 07:19:22.910192 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.910174 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c4959aeb960c8e20c88ac2de924a35b8221494-145dcd5a-dockercfg-kp896\"" Nov 04 07:19:22.928851 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.928828 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq"] Nov 04 07:19:22.929375 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.929356 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg"] Nov 04 07:19:22.992159 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.992128 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrq44\" (UniqueName: \"kubernetes.io/projected/ff1e822e-f9a7-4739-99c4-83dd0c094156-kube-api-access-nrq44\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq\" (UID: \"ff1e822e-f9a7-4739-99c4-83dd0c094156\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" Nov 04 07:19:22.992282 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.992162 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg\" (UID: \"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" Nov 04 07:19:22.992282 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.992189 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm7s5\" (UniqueName: \"kubernetes.io/projected/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-kube-api-access-rm7s5\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg\" (UID: \"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" Nov 04 07:19:22.992362 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.992287 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff1e822e-f9a7-4739-99c4-83dd0c094156-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq\" (UID: \"ff1e822e-f9a7-4739-99c4-83dd0c094156\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" Nov 04 07:19:22.992362 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.992323 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg\" (UID: \"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" Nov 04 07:19:22.992362 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:22.992343 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff1e822e-f9a7-4739-99c4-83dd0c094156-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq\" (UID: \"ff1e822e-f9a7-4739-99c4-83dd0c094156\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" Nov 04 07:19:23.093001 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.092974 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrq44\" (UniqueName: \"kubernetes.io/projected/ff1e822e-f9a7-4739-99c4-83dd0c094156-kube-api-access-nrq44\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq\" (UID: \"ff1e822e-f9a7-4739-99c4-83dd0c094156\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" Nov 04 07:19:23.093093 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.093007 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg\" (UID: \"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" Nov 04 07:19:23.093093 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.093041 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm7s5\" (UniqueName: \"kubernetes.io/projected/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-kube-api-access-rm7s5\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg\" (UID: \"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" Nov 04 07:19:23.093093 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.093071 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff1e822e-f9a7-4739-99c4-83dd0c094156-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq\" (UID: \"ff1e822e-f9a7-4739-99c4-83dd0c094156\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" Nov 04 07:19:23.093205 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.093119 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg\" (UID: \"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" Nov 04 07:19:23.093205 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.093168 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff1e822e-f9a7-4739-99c4-83dd0c094156-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq\" (UID: \"ff1e822e-f9a7-4739-99c4-83dd0c094156\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" Nov 04 07:19:23.093439 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.093405 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg\" (UID: \"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" Nov 04 07:19:23.093474 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.093434 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg\" (UID: \"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" Nov 04 07:19:23.093474 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.093462 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff1e822e-f9a7-4739-99c4-83dd0c094156-util\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq\" (UID: \"ff1e822e-f9a7-4739-99c4-83dd0c094156\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" Nov 04 07:19:23.093601 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.093585 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff1e822e-f9a7-4739-99c4-83dd0c094156-bundle\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq\" (UID: \"ff1e822e-f9a7-4739-99c4-83dd0c094156\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" Nov 04 07:19:23.105282 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.105255 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm7s5\" (UniqueName: \"kubernetes.io/projected/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-kube-api-access-rm7s5\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg\" (UID: \"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" Nov 04 07:19:23.105369 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.105302 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrq44\" (UniqueName: \"kubernetes.io/projected/ff1e822e-f9a7-4739-99c4-83dd0c094156-kube-api-access-nrq44\") pod \"c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq\" (UID: \"ff1e822e-f9a7-4739-99c4-83dd0c094156\") " pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" Nov 04 07:19:23.211850 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.211775 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" Nov 04 07:19:23.218013 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.217995 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" Nov 04 07:19:23.265411 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.265371 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn"] Nov 04 07:19:23.270588 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.269151 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.276598 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.273601 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Nov 04 07:19:23.276598 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.273615 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Nov 04 07:19:23.276598 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.273639 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Nov 04 07:19:23.276598 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.273663 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Nov 04 07:19:23.276598 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.273708 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Nov 04 07:19:23.276598 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.273711 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-lnsrk\"" Nov 04 07:19:23.276598 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.273779 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Nov 04 07:19:23.284704 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.278839 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:23.284704 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.284391 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn"] Nov 04 07:19:23.392071 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.392040 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:23.396117 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.396098 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f7cc063-10a6-471b-9169-b200fedd53de-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.396183 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.396126 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.396183 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.396150 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f7cc063-10a6-471b-9169-b200fedd53de-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.396262 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.396247 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sffsq\" (UniqueName: \"kubernetes.io/projected/5f7cc063-10a6-471b-9169-b200fedd53de-kube-api-access-sffsq\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.396318 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.396307 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.396352 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.396328 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5f7cc063-10a6-471b-9169-b200fedd53de-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.396403 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.396350 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.496356 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.496280 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg"] Nov 04 07:19:23.497414 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.497392 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.497459 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.497431 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f7cc063-10a6-471b-9169-b200fedd53de-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.497518 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.497463 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sffsq\" (UniqueName: \"kubernetes.io/projected/5f7cc063-10a6-471b-9169-b200fedd53de-kube-api-access-sffsq\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.497518 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.497500 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.499109 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.498623 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5f7cc063-10a6-471b-9169-b200fedd53de-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.499109 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.498685 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.499109 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.498739 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f7cc063-10a6-471b-9169-b200fedd53de-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.499109 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.498893 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq"] Nov 04 07:19:23.499109 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.499059 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5f7cc063-10a6-471b-9169-b200fedd53de-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.500523 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.500465 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.500712 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:19:23.500685 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ceb4eb1_34a1_4abf_9f3a_823e1e33c6f7.slice/crio-980405e1369c2f9cc21a3b3c984baefb17081cbfedc1451d239546dd0ea81cf7 WatchSource:0}: Error finding container 980405e1369c2f9cc21a3b3c984baefb17081cbfedc1451d239546dd0ea81cf7: Status 404 returned error can't find the container with id 980405e1369c2f9cc21a3b3c984baefb17081cbfedc1451d239546dd0ea81cf7 Nov 04 07:19:23.501497 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.501402 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.501596 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.501567 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.501856 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.501837 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f7cc063-10a6-471b-9169-b200fedd53de-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.503303 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:19:23.503283 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff1e822e_f9a7_4739_99c4_83dd0c094156.slice/crio-8aba281656e87c92f742705bed0780c9b717d9e57ed9c6fb55ce0463d26271e8 WatchSource:0}: Error finding container 8aba281656e87c92f742705bed0780c9b717d9e57ed9c6fb55ce0463d26271e8: Status 404 returned error can't find the container with id 8aba281656e87c92f742705bed0780c9b717d9e57ed9c6fb55ce0463d26271e8 Nov 04 07:19:23.509942 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.509921 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f7cc063-10a6-471b-9169-b200fedd53de-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.510035 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.509970 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sffsq\" (UniqueName: \"kubernetes.io/projected/5f7cc063-10a6-471b-9169-b200fedd53de-kube-api-access-sffsq\") pod \"istiod-openshift-gateway-7876bfcb74-7kvxn\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.545112 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.545080 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" event={"ID":"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7","Type":"ContainerStarted","Data":"980405e1369c2f9cc21a3b3c984baefb17081cbfedc1451d239546dd0ea81cf7"} Nov 04 07:19:23.546088 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.546063 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" event={"ID":"ff1e822e-f9a7-4739-99c4-83dd0c094156","Type":"ContainerStarted","Data":"8aba281656e87c92f742705bed0780c9b717d9e57ed9c6fb55ce0463d26271e8"} Nov 04 07:19:23.591325 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.591300 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:23.868614 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:23.868586 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn"] Nov 04 07:19:23.871643 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:19:23.871618 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f7cc063_10a6_471b_9169_b200fedd53de.slice/crio-67f51d60239b1b401a1f0951f94c3cf85e7dbe07f8fc45559d2489880e39dc3e WatchSource:0}: Error finding container 67f51d60239b1b401a1f0951f94c3cf85e7dbe07f8fc45559d2489880e39dc3e: Status 404 returned error can't find the container with id 67f51d60239b1b401a1f0951f94c3cf85e7dbe07f8fc45559d2489880e39dc3e Nov 04 07:19:24.553572 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:24.553534 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" event={"ID":"5f7cc063-10a6-471b-9169-b200fedd53de","Type":"ContainerStarted","Data":"67f51d60239b1b401a1f0951f94c3cf85e7dbe07f8fc45559d2489880e39dc3e"} Nov 04 07:19:24.557123 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:24.557085 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:27.323848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:27.323385 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:19:27.323848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:27.323468 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:19:27.569818 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:27.569781 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" event={"ID":"5f7cc063-10a6-471b-9169-b200fedd53de","Type":"ContainerStarted","Data":"8fe4fbeb21dfcf6fc9b93266d94facf688b4911ed7ad637559fa7b8f67d2c15a"} Nov 04 07:19:27.569975 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:27.569942 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:28.176107 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:28.176058 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:28.291496 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:28.291467 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:28.576461 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:28.576355 5385 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-7kvxn container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 07:19:28.576461 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:28.576428 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" podUID="5f7cc063-10a6-471b-9169-b200fedd53de" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 07:19:29.317084 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:29.317041 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:29.464368 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:29.464326 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:30.583148 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:30.583065 5385 generic.go:356] "Generic (PLEG): container finished" podID="ff1e822e-f9a7-4739-99c4-83dd0c094156" containerID="497a2ff2a9586908f18332395d902673603c53da4a39697f3a234b778066dc62" exitCode=0 Nov 04 07:19:30.583603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:30.583157 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" event={"ID":"ff1e822e-f9a7-4739-99c4-83dd0c094156","Type":"ContainerDied","Data":"497a2ff2a9586908f18332395d902673603c53da4a39697f3a234b778066dc62"} Nov 04 07:19:30.584686 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:30.584660 5385 generic.go:356] "Generic (PLEG): container finished" podID="1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7" containerID="9c16922979e63673d70022b38490cc618e8e8dc68ab22b9731a9c412700a6e48" exitCode=0 Nov 04 07:19:30.584785 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:30.584741 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" event={"ID":"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7","Type":"ContainerDied","Data":"9c16922979e63673d70022b38490cc618e8e8dc68ab22b9731a9c412700a6e48"} Nov 04 07:19:30.607641 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:30.607596 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" podStartSLOduration=4.158736958 podStartE2EDuration="7.607583183s" totalImagesPullingTime="3.448846225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:19:27.598773712 +0000 UTC m=+202.932713296" watchObservedRunningTime="2026-11-04 07:19:30.607583183 +0000 UTC m=+205.941522759" Nov 04 07:19:30.873569 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:19:30.873536 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:19:31.575994 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:31.575915 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:19:31.973923 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:31.973889 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:33.597362 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:33.597279 5385 generic.go:356] "Generic (PLEG): container finished" podID="ff1e822e-f9a7-4739-99c4-83dd0c094156" containerID="d726488ce857eb10ab5389310995241d240607c7c3491e66da873926aa9299bd" exitCode=0 Nov 04 07:19:33.597833 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:33.597373 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" event={"ID":"ff1e822e-f9a7-4739-99c4-83dd0c094156","Type":"ContainerDied","Data":"d726488ce857eb10ab5389310995241d240607c7c3491e66da873926aa9299bd"} Nov 04 07:19:33.599364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:33.599342 5385 generic.go:356] "Generic (PLEG): container finished" podID="1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7" containerID="57fd22f418f168a0cc1b78594e863d9badce7055d528f8192fea64364c278d7f" exitCode=0 Nov 04 07:19:33.599458 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:33.599406 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" event={"ID":"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7","Type":"ContainerDied","Data":"57fd22f418f168a0cc1b78594e863d9badce7055d528f8192fea64364c278d7f"} Nov 04 07:19:40.630604 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:40.630571 5385 generic.go:356] "Generic (PLEG): container finished" podID="ff1e822e-f9a7-4739-99c4-83dd0c094156" containerID="9ba2e4c279a170d7a85243799f83da3d56a5be3d2a594259876d20025f8ad2ff" exitCode=0 Nov 04 07:19:40.630941 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:40.630660 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" event={"ID":"ff1e822e-f9a7-4739-99c4-83dd0c094156","Type":"ContainerDied","Data":"9ba2e4c279a170d7a85243799f83da3d56a5be3d2a594259876d20025f8ad2ff"} Nov 04 07:19:40.632739 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:40.632717 5385 generic.go:356] "Generic (PLEG): container finished" podID="1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7" containerID="eb46746489e6212ec2c63cfd0dad05244a76b672303cc6418780339fa7b68d07" exitCode=0 Nov 04 07:19:40.632821 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:40.632778 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" event={"ID":"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7","Type":"ContainerDied","Data":"eb46746489e6212ec2c63cfd0dad05244a76b672303cc6418780339fa7b68d07"} Nov 04 07:19:41.759483 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:41.759455 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" Nov 04 07:19:41.786921 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:41.786898 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" Nov 04 07:19:41.866349 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:41.866323 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-bundle\") pod \"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7\" (UID: \"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7\") " Nov 04 07:19:41.866555 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:41.866386 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-util\" (UniqueName: \"kubernetes.io/empty-dir/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-util\") pod \"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7\" (UID: \"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7\") " Nov 04 07:19:41.866555 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:41.866439 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-kube-api-access-rm7s5\" (UniqueName: \"kubernetes.io/projected/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-kube-api-access-rm7s5\") pod \"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7\" (UID: \"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7\") " Nov 04 07:19:41.867011 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:41.866986 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-bundle" pod "1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7" (UID: "1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:19:41.868603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:41.868582 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-kube-api-access-rm7s5" pod "1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7" (UID: "1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7"). InnerVolumeSpecName "kube-api-access-rm7s5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:19:41.870354 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:41.870327 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-util" pod "1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7" (UID: "1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:19:41.967192 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:41.967100 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff1e822e-f9a7-4739-99c4-83dd0c094156-util\" (UniqueName: \"kubernetes.io/empty-dir/ff1e822e-f9a7-4739-99c4-83dd0c094156-util\") pod \"ff1e822e-f9a7-4739-99c4-83dd0c094156\" (UID: \"ff1e822e-f9a7-4739-99c4-83dd0c094156\") " Nov 04 07:19:41.967192 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:41.967157 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ff1e822e-f9a7-4739-99c4-83dd0c094156-kube-api-access-nrq44\" (UniqueName: \"kubernetes.io/projected/ff1e822e-f9a7-4739-99c4-83dd0c094156-kube-api-access-nrq44\") pod \"ff1e822e-f9a7-4739-99c4-83dd0c094156\" (UID: \"ff1e822e-f9a7-4739-99c4-83dd0c094156\") " Nov 04 07:19:41.967192 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:41.967188 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ff1e822e-f9a7-4739-99c4-83dd0c094156-bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff1e822e-f9a7-4739-99c4-83dd0c094156-bundle\") pod \"ff1e822e-f9a7-4739-99c4-83dd0c094156\" (UID: \"ff1e822e-f9a7-4739-99c4-83dd0c094156\") " Nov 04 07:19:41.967460 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:41.967369 5385 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-util\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:19:41.967460 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:41.967388 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rm7s5\" (UniqueName: \"kubernetes.io/projected/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-kube-api-access-rm7s5\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:19:41.967460 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:41.967402 5385 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7-bundle\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:19:41.967645 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:41.967626 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff1e822e-f9a7-4739-99c4-83dd0c094156-bundle" pod "ff1e822e-f9a7-4739-99c4-83dd0c094156" (UID: "ff1e822e-f9a7-4739-99c4-83dd0c094156"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:19:41.969221 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:41.969197 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff1e822e-f9a7-4739-99c4-83dd0c094156-kube-api-access-nrq44" pod "ff1e822e-f9a7-4739-99c4-83dd0c094156" (UID: "ff1e822e-f9a7-4739-99c4-83dd0c094156"). InnerVolumeSpecName "kube-api-access-nrq44". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:19:41.972321 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:41.972301 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff1e822e-f9a7-4739-99c4-83dd0c094156-util" pod "ff1e822e-f9a7-4739-99c4-83dd0c094156" (UID: "ff1e822e-f9a7-4739-99c4-83dd0c094156"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:19:42.067797 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:42.067751 5385 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ff1e822e-f9a7-4739-99c4-83dd0c094156-util\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:19:42.067797 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:42.067795 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrq44\" (UniqueName: \"kubernetes.io/projected/ff1e822e-f9a7-4739-99c4-83dd0c094156-kube-api-access-nrq44\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:19:42.067797 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:42.067805 5385 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ff1e822e-f9a7-4739-99c4-83dd0c094156-bundle\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:19:42.641524 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:42.641483 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" Nov 04 07:19:42.641524 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:42.641486 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c4959aeb960c8e20c88ac2de924a35b8221494bf772d0dcdd61893e644g98mq" event={"ID":"ff1e822e-f9a7-4739-99c4-83dd0c094156","Type":"ContainerDied","Data":"8aba281656e87c92f742705bed0780c9b717d9e57ed9c6fb55ce0463d26271e8"} Nov 04 07:19:42.641769 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:42.641553 5385 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8aba281656e87c92f742705bed0780c9b717d9e57ed9c6fb55ce0463d26271e8" Nov 04 07:19:42.643379 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:42.643352 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" event={"ID":"1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7","Type":"ContainerDied","Data":"980405e1369c2f9cc21a3b3c984baefb17081cbfedc1451d239546dd0ea81cf7"} Nov 04 07:19:42.643536 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:42.643390 5385 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="980405e1369c2f9cc21a3b3c984baefb17081cbfedc1451d239546dd0ea81cf7" Nov 04 07:19:42.643536 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:42.643364 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cvrlbg" Nov 04 07:19:48.557217 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:48.557181 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:49.707262 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:49.707225 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:57.612371 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:57.612335 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:57.721189 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:57.721159 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:58.190104 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.190075 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:58.387545 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.387498 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc"] Nov 04 07:19:58.387786 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.387774 5385 cpu_manager.go:414] "containerMap: removing container" podUID="1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7" containerName="extract" Nov 04 07:19:58.387786 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.387786 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7" containerName="extract" Nov 04 07:19:58.387934 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.387800 5385 cpu_manager.go:414] "containerMap: removing container" podUID="ff1e822e-f9a7-4739-99c4-83dd0c094156" containerName="util" Nov 04 07:19:58.387934 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.387806 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff1e822e-f9a7-4739-99c4-83dd0c094156" containerName="util" Nov 04 07:19:58.387934 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.387815 5385 cpu_manager.go:414] "containerMap: removing container" podUID="1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7" containerName="util" Nov 04 07:19:58.387934 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.387820 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7" containerName="util" Nov 04 07:19:58.387934 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.387840 5385 cpu_manager.go:414] "containerMap: removing container" podUID="ff1e822e-f9a7-4739-99c4-83dd0c094156" containerName="extract" Nov 04 07:19:58.387934 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.387846 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff1e822e-f9a7-4739-99c4-83dd0c094156" containerName="extract" Nov 04 07:19:58.387934 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.387862 5385 cpu_manager.go:414] "containerMap: removing container" podUID="ff1e822e-f9a7-4739-99c4-83dd0c094156" containerName="pull" Nov 04 07:19:58.387934 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.387869 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ff1e822e-f9a7-4739-99c4-83dd0c094156" containerName="pull" Nov 04 07:19:58.387934 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.387881 5385 cpu_manager.go:414] "containerMap: removing container" podUID="1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7" containerName="pull" Nov 04 07:19:58.387934 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.387887 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7" containerName="pull" Nov 04 07:19:58.388316 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.387945 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ceb4eb1-34a1-4abf-9f3a-823e1e33c6f7" containerName="extract" Nov 04 07:19:58.388316 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.387960 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="ff1e822e-f9a7-4739-99c4-83dd0c094156" containerName="extract" Nov 04 07:19:58.389717 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.389701 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.409446 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.409425 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\"" Nov 04 07:19:58.409579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.409529 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\"" Nov 04 07:19:58.409579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.409549 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\"" Nov 04 07:19:58.409579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.409560 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\"" Nov 04 07:19:58.409700 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.409602 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-nhb6m\"" Nov 04 07:19:58.409700 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.409672 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\"" Nov 04 07:19:58.427031 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.427004 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc"] Nov 04 07:19:58.483548 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.483459 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce14b071-5b23-4026-ae18-122fc62ebee8-manifests\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.483548 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.483488 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce14b071-5b23-4026-ae18-122fc62ebee8-tmp\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.483734 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.483609 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce14b071-5b23-4026-ae18-122fc62ebee8-metrics-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.483734 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.483649 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce14b071-5b23-4026-ae18-122fc62ebee8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.483734 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.483686 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qxlf\" (UniqueName: \"kubernetes.io/projected/ce14b071-5b23-4026-ae18-122fc62ebee8-kube-api-access-8qxlf\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.483831 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.483777 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce14b071-5b23-4026-ae18-122fc62ebee8-webhook-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.518050 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.518023 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:58.584206 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.584179 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce14b071-5b23-4026-ae18-122fc62ebee8-metrics-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.584335 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.584215 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce14b071-5b23-4026-ae18-122fc62ebee8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.584398 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.584332 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qxlf\" (UniqueName: \"kubernetes.io/projected/ce14b071-5b23-4026-ae18-122fc62ebee8-kube-api-access-8qxlf\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.584398 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.584384 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce14b071-5b23-4026-ae18-122fc62ebee8-webhook-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.584493 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.584420 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce14b071-5b23-4026-ae18-122fc62ebee8-manifests\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.584493 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.584442 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce14b071-5b23-4026-ae18-122fc62ebee8-tmp\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.584924 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.584898 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ce14b071-5b23-4026-ae18-122fc62ebee8-manifests\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.586716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.586693 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ce14b071-5b23-4026-ae18-122fc62ebee8-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.586812 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.586735 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ce14b071-5b23-4026-ae18-122fc62ebee8-tmp\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.586932 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.586912 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce14b071-5b23-4026-ae18-122fc62ebee8-webhook-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.586971 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.586913 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce14b071-5b23-4026-ae18-122fc62ebee8-metrics-cert\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.597793 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.597773 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qxlf\" (UniqueName: \"kubernetes.io/projected/ce14b071-5b23-4026-ae18-122fc62ebee8-kube-api-access-8qxlf\") pod \"opendatahub-operator-controller-manager-8f94f88d4-2xfwc\" (UID: \"ce14b071-5b23-4026-ae18-122fc62ebee8\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.650416 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.650391 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:58.701751 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.701727 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:19:58.777863 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.777833 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:58.877385 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.877355 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:19:58.999612 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:58.999580 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc"] Nov 04 07:19:59.003231 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:19:59.003198 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce14b071_5b23_4026_ae18_122fc62ebee8.slice/crio-98ec2ec6f35946ea7fdfe75685e00f1a17af569e0cd24b5c91cc81ff9d6c6659 WatchSource:0}: Error finding container 98ec2ec6f35946ea7fdfe75685e00f1a17af569e0cd24b5c91cc81ff9d6c6659: Status 404 returned error can't find the container with id 98ec2ec6f35946ea7fdfe75685e00f1a17af569e0cd24b5c91cc81ff9d6c6659 Nov 04 07:19:59.707908 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:19:59.707858 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" event={"ID":"ce14b071-5b23-4026-ae18-122fc62ebee8","Type":"ContainerStarted","Data":"98ec2ec6f35946ea7fdfe75685e00f1a17af569e0cd24b5c91cc81ff9d6c6659"} Nov 04 07:20:00.798071 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:00.798031 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:00.876231 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:20:00.876197 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:20:01.415433 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:01.415397 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:02.721925 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:02.721889 5385 generic.go:356] "Generic (PLEG): container finished" podID="ce14b071-5b23-4026-ae18-122fc62ebee8" containerID="cd21aa9f4e3b489a55dc7bc98d6a01ec0d237eeafc0c7e7ad27c4552d36b277a" exitCode=0 Nov 04 07:20:02.722299 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:02.721979 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" event={"ID":"ce14b071-5b23-4026-ae18-122fc62ebee8","Type":"ContainerDied","Data":"cd21aa9f4e3b489a55dc7bc98d6a01ec0d237eeafc0c7e7ad27c4552d36b277a"} Nov 04 07:20:03.727872 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:03.727833 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" event={"ID":"ce14b071-5b23-4026-ae18-122fc62ebee8","Type":"ContainerStarted","Data":"dd16f6c69213a0f03ab52220e5468a48dc4dceb953dcfd958ce6416c60f51763"} Nov 04 07:20:03.728363 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:03.727970 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:20:03.773232 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:03.773185 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" podStartSLOduration=1.9358566069999998 podStartE2EDuration="5.773171546s" totalImagesPullingTime="3.353547033s" totalInitContainerRuntime="483.767906ms" isStatefulPod=false podCreationTimestamp="2026-11-04 07:19:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:20:03.771874224 +0000 UTC m=+239.105813811" watchObservedRunningTime="2026-11-04 07:20:03.773171546 +0000 UTC m=+239.107111121" Nov 04 07:20:05.429103 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:05.429067 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:06.591891 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:06.591838 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:08.253921 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:08.253873 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:08.590461 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:08.590384 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:08.691002 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:08.690966 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:10.096765 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:10.096730 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:10.217882 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:10.217844 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:10.818913 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:10.818868 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:10.921933 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:10.921903 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:11.629371 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:11.629317 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:12.150447 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:12.150411 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:12.560949 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:12.560833 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:13.070467 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:13.070436 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:13.211320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:13.211288 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:14.734829 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:14.734798 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8f94f88d4-2xfwc" Nov 04 07:20:15.321157 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:15.321109 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:15.444155 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:15.444119 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:15.659761 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:15.659726 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:18.063962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:18.063924 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:18.171667 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:18.171628 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:18.481430 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:18.481398 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:19.658592 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:19.658551 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:19.998538 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:19.998446 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:20.269841 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:20.269757 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:20.439877 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:20.439848 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:20.813085 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:20.813050 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:20.958416 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:20.958384 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:21.244829 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:21.244796 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:27.404182 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:27.404140 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:27.510761 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:27.510732 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:28.463879 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:28.463847 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:28.566157 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:28.566122 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:30.857356 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:30.857317 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:30.874723 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:20:30.874692 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:20:31.004915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.004878 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:31.157662 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.157633 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7"] Nov 04 07:20:31.160700 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.160683 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7" Nov 04 07:20:31.178045 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.178021 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Nov 04 07:20:31.178045 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.178034 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Nov 04 07:20:31.178961 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.178947 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-mvxk2\"" Nov 04 07:20:31.180369 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.180345 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7"] Nov 04 07:20:31.235738 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.235707 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:31.247889 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.247861 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5bvx\" (UniqueName: \"kubernetes.io/projected/ebcd9bc5-c463-4550-820d-6648936b12a3-kube-api-access-j5bvx\") pod \"kube-auth-proxy-589fcc7ff4-dcwx7\" (UID: \"ebcd9bc5-c463-4550-820d-6648936b12a3\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7" Nov 04 07:20:31.248025 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.247918 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebcd9bc5-c463-4550-820d-6648936b12a3-tls-certs\") pod \"kube-auth-proxy-589fcc7ff4-dcwx7\" (UID: \"ebcd9bc5-c463-4550-820d-6648936b12a3\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7" Nov 04 07:20:31.248076 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.248024 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebcd9bc5-c463-4550-820d-6648936b12a3-tmp\") pod \"kube-auth-proxy-589fcc7ff4-dcwx7\" (UID: \"ebcd9bc5-c463-4550-820d-6648936b12a3\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7" Nov 04 07:20:31.348817 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.348784 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebcd9bc5-c463-4550-820d-6648936b12a3-tls-certs\") pod \"kube-auth-proxy-589fcc7ff4-dcwx7\" (UID: \"ebcd9bc5-c463-4550-820d-6648936b12a3\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7" Nov 04 07:20:31.348994 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.348947 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebcd9bc5-c463-4550-820d-6648936b12a3-tmp\") pod \"kube-auth-proxy-589fcc7ff4-dcwx7\" (UID: \"ebcd9bc5-c463-4550-820d-6648936b12a3\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7" Nov 04 07:20:31.349064 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:20:31.349000 5385 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Nov 04 07:20:31.349064 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.349036 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5bvx\" (UniqueName: \"kubernetes.io/projected/ebcd9bc5-c463-4550-820d-6648936b12a3-kube-api-access-j5bvx\") pod \"kube-auth-proxy-589fcc7ff4-dcwx7\" (UID: \"ebcd9bc5-c463-4550-820d-6648936b12a3\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7" Nov 04 07:20:31.349164 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:20:31.349073 5385 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebcd9bc5-c463-4550-820d-6648936b12a3-tls-certs podName:ebcd9bc5-c463-4550-820d-6648936b12a3 nodeName:}" failed. No retries permitted until 2026-11-04 07:20:31.849056059 +0000 UTC m=+267.182995648 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ebcd9bc5-c463-4550-820d-6648936b12a3-tls-certs") pod "kube-auth-proxy-589fcc7ff4-dcwx7" (UID: "ebcd9bc5-c463-4550-820d-6648936b12a3") : secret "kube-auth-proxy-tls" not found Nov 04 07:20:31.351521 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.351477 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ebcd9bc5-c463-4550-820d-6648936b12a3-tmp\") pod \"kube-auth-proxy-589fcc7ff4-dcwx7\" (UID: \"ebcd9bc5-c463-4550-820d-6648936b12a3\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7" Nov 04 07:20:31.374917 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.374892 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5bvx\" (UniqueName: \"kubernetes.io/projected/ebcd9bc5-c463-4550-820d-6648936b12a3-kube-api-access-j5bvx\") pod \"kube-auth-proxy-589fcc7ff4-dcwx7\" (UID: \"ebcd9bc5-c463-4550-820d-6648936b12a3\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7" Nov 04 07:20:31.376702 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.376679 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:31.529527 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.529433 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:31.635739 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.635669 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:31.853564 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.853487 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebcd9bc5-c463-4550-820d-6648936b12a3-tls-certs\") pod \"kube-auth-proxy-589fcc7ff4-dcwx7\" (UID: \"ebcd9bc5-c463-4550-820d-6648936b12a3\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7" Nov 04 07:20:31.855920 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:31.855896 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ebcd9bc5-c463-4550-820d-6648936b12a3-tls-certs\") pod \"kube-auth-proxy-589fcc7ff4-dcwx7\" (UID: \"ebcd9bc5-c463-4550-820d-6648936b12a3\") " pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7" Nov 04 07:20:32.072088 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:32.072054 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7" Nov 04 07:20:32.343253 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:32.343230 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7"] Nov 04 07:20:32.345652 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:20:32.345625 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebcd9bc5_c463_4550_820d_6648936b12a3.slice/crio-ad1d98222d4bd4575be2b3a45d725fe304e4f5282f8fa8243e555fc591e70b0e WatchSource:0}: Error finding container ad1d98222d4bd4575be2b3a45d725fe304e4f5282f8fa8243e555fc591e70b0e: Status 404 returned error can't find the container with id ad1d98222d4bd4575be2b3a45d725fe304e4f5282f8fa8243e555fc591e70b0e Nov 04 07:20:32.826076 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:32.826035 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7" event={"ID":"ebcd9bc5-c463-4550-820d-6648936b12a3","Type":"ContainerStarted","Data":"ad1d98222d4bd4575be2b3a45d725fe304e4f5282f8fa8243e555fc591e70b0e"} Nov 04 07:20:32.928736 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:32.928698 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:33.405055 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:33.405021 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:34.623641 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.623604 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv"] Nov 04 07:20:34.627369 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.627348 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" Nov 04 07:20:34.632596 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.632558 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-module-controller-manager-dockercfg-t5jnw\"" Nov 04 07:20:34.632733 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.632593 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Nov 04 07:20:34.632733 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.632650 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Nov 04 07:20:34.635610 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.635591 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"odh-trusted-ca-bundle\"" Nov 04 07:20:34.642632 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.642609 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv"] Nov 04 07:20:34.779755 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.779723 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a23d550f-ddb6-4615-befd-cdaf1ed4d185-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-zzdxv\" (UID: \"a23d550f-ddb6-4615-befd-cdaf1ed4d185\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" Nov 04 07:20:34.779877 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.779787 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a23d550f-ddb6-4615-befd-cdaf1ed4d185-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-zzdxv\" (UID: \"a23d550f-ddb6-4615-befd-cdaf1ed4d185\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" Nov 04 07:20:34.779877 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.779835 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a23d550f-ddb6-4615-befd-cdaf1ed4d185-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-zzdxv\" (UID: \"a23d550f-ddb6-4615-befd-cdaf1ed4d185\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" Nov 04 07:20:34.779989 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.779974 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6m9jn\" (UniqueName: \"kubernetes.io/projected/a23d550f-ddb6-4615-befd-cdaf1ed4d185-kube-api-access-6m9jn\") pod \"kserve-module-controller-manager-7566cf5fc8-zzdxv\" (UID: \"a23d550f-ddb6-4615-befd-cdaf1ed4d185\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" Nov 04 07:20:34.834286 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.834257 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7" event={"ID":"ebcd9bc5-c463-4550-820d-6648936b12a3","Type":"ContainerStarted","Data":"65492058a602865c74b38f78950a183cc2f91864f4ea69a1522d203db062cfbc"} Nov 04 07:20:34.854650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.854608 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-589fcc7ff4-dcwx7" podStartSLOduration=1.5179545239999999 podStartE2EDuration="3.854595348s" totalImagesPullingTime="2.336640824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:20:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:20:34.853353151 +0000 UTC m=+270.187292754" watchObservedRunningTime="2026-11-04 07:20:34.854595348 +0000 UTC m=+270.188534926" Nov 04 07:20:34.880904 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.880843 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6m9jn\" (UniqueName: \"kubernetes.io/projected/a23d550f-ddb6-4615-befd-cdaf1ed4d185-kube-api-access-6m9jn\") pod \"kserve-module-controller-manager-7566cf5fc8-zzdxv\" (UID: \"a23d550f-ddb6-4615-befd-cdaf1ed4d185\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" Nov 04 07:20:34.881007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.880987 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a23d550f-ddb6-4615-befd-cdaf1ed4d185-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-zzdxv\" (UID: \"a23d550f-ddb6-4615-befd-cdaf1ed4d185\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" Nov 04 07:20:34.881047 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.881028 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a23d550f-ddb6-4615-befd-cdaf1ed4d185-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-zzdxv\" (UID: \"a23d550f-ddb6-4615-befd-cdaf1ed4d185\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" Nov 04 07:20:34.881091 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.881043 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a23d550f-ddb6-4615-befd-cdaf1ed4d185-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-zzdxv\" (UID: \"a23d550f-ddb6-4615-befd-cdaf1ed4d185\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" Nov 04 07:20:34.881516 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.881447 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a23d550f-ddb6-4615-befd-cdaf1ed4d185-manifests\") pod \"kserve-module-controller-manager-7566cf5fc8-zzdxv\" (UID: \"a23d550f-ddb6-4615-befd-cdaf1ed4d185\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" Nov 04 07:20:34.881516 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.881484 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a23d550f-ddb6-4615-befd-cdaf1ed4d185-tmp\") pod \"kserve-module-controller-manager-7566cf5fc8-zzdxv\" (UID: \"a23d550f-ddb6-4615-befd-cdaf1ed4d185\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" Nov 04 07:20:34.882059 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.882038 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a23d550f-ddb6-4615-befd-cdaf1ed4d185-trusted-ca\") pod \"kserve-module-controller-manager-7566cf5fc8-zzdxv\" (UID: \"a23d550f-ddb6-4615-befd-cdaf1ed4d185\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" Nov 04 07:20:34.896259 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.896214 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6m9jn\" (UniqueName: \"kubernetes.io/projected/a23d550f-ddb6-4615-befd-cdaf1ed4d185-kube-api-access-6m9jn\") pod \"kserve-module-controller-manager-7566cf5fc8-zzdxv\" (UID: \"a23d550f-ddb6-4615-befd-cdaf1ed4d185\") " pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" Nov 04 07:20:34.940265 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:34.940241 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" Nov 04 07:20:35.065807 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:35.065776 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:35.204382 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:35.204312 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:35.219577 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:35.219549 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv"] Nov 04 07:20:35.223396 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:20:35.223369 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda23d550f_ddb6_4615_befd_cdaf1ed4d185.slice/crio-5a9bff9af449a1fb7cbb1de779cca75586d57c3e1504599266864d41e206b7ab WatchSource:0}: Error finding container 5a9bff9af449a1fb7cbb1de779cca75586d57c3e1504599266864d41e206b7ab: Status 404 returned error can't find the container with id 5a9bff9af449a1fb7cbb1de779cca75586d57c3e1504599266864d41e206b7ab Nov 04 07:20:35.840533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:35.840480 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" event={"ID":"a23d550f-ddb6-4615-befd-cdaf1ed4d185","Type":"ContainerStarted","Data":"5a9bff9af449a1fb7cbb1de779cca75586d57c3e1504599266864d41e206b7ab"} Nov 04 07:20:36.846021 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:36.845259 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" event={"ID":"a23d550f-ddb6-4615-befd-cdaf1ed4d185","Type":"ContainerStarted","Data":"0e73530bfabe5dbb41b5595e7ec6a22401424469402ee4162ef83e38a3cf5cc3"} Nov 04 07:20:36.846021 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:36.845969 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" Nov 04 07:20:36.869368 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:36.869322 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" podStartSLOduration=1.490027661 podStartE2EDuration="2.869309491s" totalImagesPullingTime="1.37928183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:20:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:20:36.869005564 +0000 UTC m=+272.202945149" watchObservedRunningTime="2026-11-04 07:20:36.869309491 +0000 UTC m=+272.203249069" Nov 04 07:20:39.977426 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:39.977389 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:40.110789 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:40.110757 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:45.670310 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:45.670274 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:45.770176 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:45.770145 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:46.945436 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:46.945403 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:47.160375 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:47.160344 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:50.498945 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:50.498879 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:50.595872 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:50.595833 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:54.515466 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:54.515407 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:54.619042 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:54.619005 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:58.853634 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:58.853604 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-module-controller-manager-7566cf5fc8-zzdxv" Nov 04 07:20:59.362696 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:59.362664 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:20:59.480126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:20:59.480085 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:21:00.873117 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:21:00.873077 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:21:15.361496 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:15.361461 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:21:26.894460 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:26.894343 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:21:27.001142 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.001113 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:21:27.026475 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.026445 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-clnnz"] Nov 04 07:21:27.032623 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.032598 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-clnnz" Nov 04 07:21:27.043692 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.043665 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-clnnz"] Nov 04 07:21:27.071823 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.071799 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Nov 04 07:21:27.072121 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.072099 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-5k2c8\"" Nov 04 07:21:27.072412 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.072384 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1444ffa-7440-4c39-82d0-e2fabf7cf4e3-cert\") pod \"kserve-controller-manager-776cc66d47-clnnz\" (UID: \"a1444ffa-7440-4c39-82d0-e2fabf7cf4e3\") " pod="opendatahub/kserve-controller-manager-776cc66d47-clnnz" Nov 04 07:21:27.072523 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.072487 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjtks\" (UniqueName: \"kubernetes.io/projected/a1444ffa-7440-4c39-82d0-e2fabf7cf4e3-kube-api-access-zjtks\") pod \"kserve-controller-manager-776cc66d47-clnnz\" (UID: \"a1444ffa-7440-4c39-82d0-e2fabf7cf4e3\") " pod="opendatahub/kserve-controller-manager-776cc66d47-clnnz" Nov 04 07:21:27.129943 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.129911 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:21:27.173537 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.173451 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjtks\" (UniqueName: \"kubernetes.io/projected/a1444ffa-7440-4c39-82d0-e2fabf7cf4e3-kube-api-access-zjtks\") pod \"kserve-controller-manager-776cc66d47-clnnz\" (UID: \"a1444ffa-7440-4c39-82d0-e2fabf7cf4e3\") " pod="opendatahub/kserve-controller-manager-776cc66d47-clnnz" Nov 04 07:21:27.173537 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.173530 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1444ffa-7440-4c39-82d0-e2fabf7cf4e3-cert\") pod \"kserve-controller-manager-776cc66d47-clnnz\" (UID: \"a1444ffa-7440-4c39-82d0-e2fabf7cf4e3\") " pod="opendatahub/kserve-controller-manager-776cc66d47-clnnz" Nov 04 07:21:27.176057 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.176035 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a1444ffa-7440-4c39-82d0-e2fabf7cf4e3-cert\") pod \"kserve-controller-manager-776cc66d47-clnnz\" (UID: \"a1444ffa-7440-4c39-82d0-e2fabf7cf4e3\") " pod="opendatahub/kserve-controller-manager-776cc66d47-clnnz" Nov 04 07:21:27.195606 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.195581 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjtks\" (UniqueName: \"kubernetes.io/projected/a1444ffa-7440-4c39-82d0-e2fabf7cf4e3-kube-api-access-zjtks\") pod \"kserve-controller-manager-776cc66d47-clnnz\" (UID: \"a1444ffa-7440-4c39-82d0-e2fabf7cf4e3\") " pod="opendatahub/kserve-controller-manager-776cc66d47-clnnz" Nov 04 07:21:27.255036 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.255006 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:21:27.343203 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.343150 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv"] Nov 04 07:21:27.345033 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.345010 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-clnnz" Nov 04 07:21:27.346481 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.346465 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.362428 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.362401 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv"] Nov 04 07:21:27.387793 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.387768 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:21:27.476223 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.476127 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.476223 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.476160 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cbtb\" (UniqueName: \"kubernetes.io/projected/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-kube-api-access-9cbtb\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.476223 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.476185 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.476458 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.476247 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.476458 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.476337 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.476458 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.476369 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.476641 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.476529 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.577182 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.577145 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.577182 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.577182 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cbtb\" (UniqueName: \"kubernetes.io/projected/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-kube-api-access-9cbtb\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.577388 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.577208 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.577388 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.577233 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.577388 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.577286 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.577388 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.577333 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.577631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.577413 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.578171 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.578147 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.579568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.579544 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.579695 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.579677 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.580033 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.580012 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.580108 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.580082 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.597190 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.597171 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.599081 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.599047 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cbtb\" (UniqueName: \"kubernetes.io/projected/26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3-kube-api-access-9cbtb\") pod \"istiod-openshift-gateway-7ddd9cfdc8-d7xzv\" (UID: \"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.637660 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.637636 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-clnnz"] Nov 04 07:21:27.640682 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:21:27.640660 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1444ffa_7440_4c39_82d0_e2fabf7cf4e3.slice/crio-ac8766cc9a45c66191d84d3e4371c6b30929779f47e543f5a7381769b44294a2 WatchSource:0}: Error finding container ac8766cc9a45c66191d84d3e4371c6b30929779f47e543f5a7381769b44294a2: Status 404 returned error can't find the container with id ac8766cc9a45c66191d84d3e4371c6b30929779f47e543f5a7381769b44294a2 Nov 04 07:21:27.674582 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.674560 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:27.955941 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.955907 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv"] Nov 04 07:21:27.958656 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:21:27.958629 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26d70d1d_8ef8_4caf_942d_5c6efb7b3fe3.slice/crio-5bd793df20e860d16dc136939a9729b0e86b4c67b02f98421f8b1c6a008c0605 WatchSource:0}: Error finding container 5bd793df20e860d16dc136939a9729b0e86b4c67b02f98421f8b1c6a008c0605: Status 404 returned error can't find the container with id 5bd793df20e860d16dc136939a9729b0e86b4c67b02f98421f8b1c6a008c0605 Nov 04 07:21:27.960724 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.960692 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:21:27.960810 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:27.960756 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:21:28.024121 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:28.024091 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-clnnz" event={"ID":"a1444ffa-7440-4c39-82d0-e2fabf7cf4e3","Type":"ContainerStarted","Data":"ac8766cc9a45c66191d84d3e4371c6b30929779f47e543f5a7381769b44294a2"} Nov 04 07:21:28.025986 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:28.025949 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" event={"ID":"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3","Type":"ContainerStarted","Data":"aabc91d2713f7730e03575fffdeb2c57e7aee8aa76c4cd907c98afd5730f2cff"} Nov 04 07:21:28.026093 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:28.026000 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" event={"ID":"26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3","Type":"ContainerStarted","Data":"5bd793df20e860d16dc136939a9729b0e86b4c67b02f98421f8b1c6a008c0605"} Nov 04 07:21:28.026093 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:28.026063 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:28.058742 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:28.058692 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" podStartSLOduration=1.058677922 podStartE2EDuration="1.058677922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:21:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:21:28.057662565 +0000 UTC m=+323.391602152" watchObservedRunningTime="2026-11-04 07:21:28.058677922 +0000 UTC m=+323.392617499" Nov 04 07:21:28.405652 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:28.405613 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:21:28.514132 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:28.514095 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:21:28.709846 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:28.709760 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:21:29.032744 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.032675 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-d7xzv" Nov 04 07:21:29.115385 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.115352 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn"] Nov 04 07:21:29.115611 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.115586 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" podUID="5f7cc063-10a6-471b-9169-b200fedd53de" containerName="discovery" containerID="cri-o://8fe4fbeb21dfcf6fc9b93266d94facf688b4911ed7ad637559fa7b8f67d2c15a" gracePeriod=30 Nov 04 07:21:29.367701 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.367674 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:21:29.495253 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.495208 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-cacerts\" (UniqueName: \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-cacerts\") pod \"5f7cc063-10a6-471b-9169-b200fedd53de\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " Nov 04 07:21:29.495455 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.495278 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-istio-kubeconfig\") pod \"5f7cc063-10a6-471b-9169-b200fedd53de\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " Nov 04 07:21:29.495662 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.495632 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f7cc063-10a6-471b-9169-b200fedd53de-istio-token\" (UniqueName: \"kubernetes.io/projected/5f7cc063-10a6-471b-9169-b200fedd53de-istio-token\") pod \"5f7cc063-10a6-471b-9169-b200fedd53de\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " Nov 04 07:21:29.495763 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.495681 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5f7cc063-10a6-471b-9169-b200fedd53de-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5f7cc063-10a6-471b-9169-b200fedd53de-istio-csr-ca-configmap\") pod \"5f7cc063-10a6-471b-9169-b200fedd53de\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " Nov 04 07:21:29.495763 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.495729 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-istio-csr-dns-cert\") pod \"5f7cc063-10a6-471b-9169-b200fedd53de\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " Nov 04 07:21:29.495868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.495800 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f7cc063-10a6-471b-9169-b200fedd53de-kube-api-access-sffsq\" (UniqueName: \"kubernetes.io/projected/5f7cc063-10a6-471b-9169-b200fedd53de-kube-api-access-sffsq\") pod \"5f7cc063-10a6-471b-9169-b200fedd53de\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " Nov 04 07:21:29.495868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.495838 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f7cc063-10a6-471b-9169-b200fedd53de-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f7cc063-10a6-471b-9169-b200fedd53de-local-certs\") pod \"5f7cc063-10a6-471b-9169-b200fedd53de\" (UID: \"5f7cc063-10a6-471b-9169-b200fedd53de\") " Nov 04 07:21:29.496342 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.496314 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f7cc063-10a6-471b-9169-b200fedd53de-istio-csr-ca-configmap" pod "5f7cc063-10a6-471b-9169-b200fedd53de" (UID: "5f7cc063-10a6-471b-9169-b200fedd53de"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:21:29.498480 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.498449 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f7cc063-10a6-471b-9169-b200fedd53de-istio-token" pod "5f7cc063-10a6-471b-9169-b200fedd53de" (UID: "5f7cc063-10a6-471b-9169-b200fedd53de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:21:29.498717 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.498491 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-cacerts" pod "5f7cc063-10a6-471b-9169-b200fedd53de" (UID: "5f7cc063-10a6-471b-9169-b200fedd53de"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:21:29.498717 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.498560 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-istio-kubeconfig" pod "5f7cc063-10a6-471b-9169-b200fedd53de" (UID: "5f7cc063-10a6-471b-9169-b200fedd53de"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:21:29.498717 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.498680 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f7cc063-10a6-471b-9169-b200fedd53de-local-certs" pod "5f7cc063-10a6-471b-9169-b200fedd53de" (UID: "5f7cc063-10a6-471b-9169-b200fedd53de"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:21:29.499037 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.499008 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f7cc063-10a6-471b-9169-b200fedd53de-kube-api-access-sffsq" pod "5f7cc063-10a6-471b-9169-b200fedd53de" (UID: "5f7cc063-10a6-471b-9169-b200fedd53de"). InnerVolumeSpecName "kube-api-access-sffsq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:21:29.499642 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.499617 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-istio-csr-dns-cert" pod "5f7cc063-10a6-471b-9169-b200fedd53de" (UID: "5f7cc063-10a6-471b-9169-b200fedd53de"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:21:29.597127 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.597103 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sffsq\" (UniqueName: \"kubernetes.io/projected/5f7cc063-10a6-471b-9169-b200fedd53de-kube-api-access-sffsq\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:21:29.597127 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.597129 5385 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f7cc063-10a6-471b-9169-b200fedd53de-local-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:21:29.597332 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.597139 5385 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-cacerts\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:21:29.597332 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.597148 5385 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-istio-kubeconfig\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:21:29.597332 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.597156 5385 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f7cc063-10a6-471b-9169-b200fedd53de-istio-token\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:21:29.597332 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.597165 5385 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5f7cc063-10a6-471b-9169-b200fedd53de-istio-csr-ca-configmap\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:21:29.597332 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:29.597174 5385 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5f7cc063-10a6-471b-9169-b200fedd53de-istio-csr-dns-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:21:30.041225 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:30.041193 5385 generic.go:356] "Generic (PLEG): container finished" podID="5f7cc063-10a6-471b-9169-b200fedd53de" containerID="8fe4fbeb21dfcf6fc9b93266d94facf688b4911ed7ad637559fa7b8f67d2c15a" exitCode=0 Nov 04 07:21:30.041793 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:30.041248 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" Nov 04 07:21:30.041793 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:30.041273 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" event={"ID":"5f7cc063-10a6-471b-9169-b200fedd53de","Type":"ContainerDied","Data":"8fe4fbeb21dfcf6fc9b93266d94facf688b4911ed7ad637559fa7b8f67d2c15a"} Nov 04 07:21:30.041793 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:30.041346 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn" event={"ID":"5f7cc063-10a6-471b-9169-b200fedd53de","Type":"ContainerDied","Data":"67f51d60239b1b401a1f0951f94c3cf85e7dbe07f8fc45559d2489880e39dc3e"} Nov 04 07:21:30.041793 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:30.041372 5385 scope.go:122] "RemoveContainer" containerID="8fe4fbeb21dfcf6fc9b93266d94facf688b4911ed7ad637559fa7b8f67d2c15a" Nov 04 07:21:30.042978 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:30.042947 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-clnnz" event={"ID":"a1444ffa-7440-4c39-82d0-e2fabf7cf4e3","Type":"ContainerStarted","Data":"b8e26fa3172011f5399a2615fe372c96aa4a0cdb9b3805b87c4831dca6719ccc"} Nov 04 07:21:30.050623 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:30.050605 5385 scope.go:122] "RemoveContainer" containerID="8fe4fbeb21dfcf6fc9b93266d94facf688b4911ed7ad637559fa7b8f67d2c15a" Nov 04 07:21:30.050896 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:21:30.050877 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fe4fbeb21dfcf6fc9b93266d94facf688b4911ed7ad637559fa7b8f67d2c15a\": container with ID starting with 8fe4fbeb21dfcf6fc9b93266d94facf688b4911ed7ad637559fa7b8f67d2c15a not found: ID does not exist" containerID="8fe4fbeb21dfcf6fc9b93266d94facf688b4911ed7ad637559fa7b8f67d2c15a" Nov 04 07:21:30.050959 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:30.050907 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fe4fbeb21dfcf6fc9b93266d94facf688b4911ed7ad637559fa7b8f67d2c15a"} err="failed to get container status \"8fe4fbeb21dfcf6fc9b93266d94facf688b4911ed7ad637559fa7b8f67d2c15a\": rpc error: code = NotFound desc = could not find container \"8fe4fbeb21dfcf6fc9b93266d94facf688b4911ed7ad637559fa7b8f67d2c15a\": container with ID starting with 8fe4fbeb21dfcf6fc9b93266d94facf688b4911ed7ad637559fa7b8f67d2c15a not found: ID does not exist" Nov 04 07:21:30.087882 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:30.087649 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-clnnz" podStartSLOduration=1.134714509 podStartE2EDuration="3.087634975s" totalImagesPullingTime="1.952920466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:21:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:21:30.087019556 +0000 UTC m=+325.420959142" watchObservedRunningTime="2026-11-04 07:21:30.087634975 +0000 UTC m=+325.421574560" Nov 04 07:21:30.128949 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:30.128923 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn"] Nov 04 07:21:30.138143 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:30.138118 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-7kvxn"] Nov 04 07:21:30.810333 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:30.810300 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f7cc063-10a6-471b-9169-b200fedd53de" path="/var/lib/kubelet/pods/5f7cc063-10a6-471b-9169-b200fedd53de/volumes" Nov 04 07:21:30.873553 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:21:30.873515 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:21:31.047971 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:31.047942 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-clnnz" Nov 04 07:21:34.734806 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:34.734778 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:21:34.854456 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:21:34.854423 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:22:00.873621 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:22:00.873586 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:22:02.056633 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:22:02.056604 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-clnnz" Nov 04 07:22:07.086373 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:22:07.086339 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:22:07.180543 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:22:07.180496 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:22:08.746066 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:22:08.746032 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:22:08.840553 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:22:08.840520 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:22:15.406402 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:22:15.406364 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:22:30.874099 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:22:30.874061 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:22:47.796915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:22:47.796861 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:22:47.896102 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:22:47.896067 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:23:00.873284 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:23:00.873249 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:23:15.461009 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:23:15.460972 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:23:30.873696 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:23:30.873660 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:24:00.874043 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:24:00.873960 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:24:12.137275 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:24:12.137238 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:24:12.388854 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:24:12.388767 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:24:15.559912 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:24:15.559876 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:24:30.873653 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:24:30.873616 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:25:00.873176 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:25:00.873142 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:25:02.017577 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:25:02.017537 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:25:10.729952 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:25:10.729914 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:25:15.677537 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:25:15.677489 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:25:26.771199 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:25:26.771163 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:25:30.873692 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:25:30.873654 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:26:00.874287 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:26:00.874249 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:26:15.638669 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:26:15.638630 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:26:30.873677 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:26:30.873623 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:27:00.873774 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:27:00.873696 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:27:15.784581 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:27:15.784538 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:27:30.873051 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:27:30.873015 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:28:00.436783 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:28:00.436748 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:28:00.874496 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:28:00.874452 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:28:15.788696 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:28:15.788619 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:28:30.873907 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:28:30.873869 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:29:00.873843 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:29:00.873804 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:29:15.818761 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:29:15.818728 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:29:30.875472 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:29:30.875437 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:30:00.874734 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:30:00.874629 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:30:16.021003 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:30:16.020961 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:30:30.873662 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:30:30.873622 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:31:00.873991 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:31:00.873956 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:31:09.157233 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.157196 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-94g7x"] Nov 04 07:31:09.157761 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.157474 5385 cpu_manager.go:414] "containerMap: removing container" podUID="5f7cc063-10a6-471b-9169-b200fedd53de" containerName="discovery" Nov 04 07:31:09.157761 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.157486 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f7cc063-10a6-471b-9169-b200fedd53de" containerName="discovery" Nov 04 07:31:09.157761 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.157565 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f7cc063-10a6-471b-9169-b200fedd53de" containerName="discovery" Nov 04 07:31:09.160457 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.160438 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" Nov 04 07:31:09.161924 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.161902 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:09.167074 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.166793 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Nov 04 07:31:09.169589 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.169573 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Nov 04 07:31:09.169708 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.169622 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Nov 04 07:31:09.171153 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.171133 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-888hp\"" Nov 04 07:31:09.172391 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.172368 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-94g7x"] Nov 04 07:31:09.260364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.260320 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-94g7x"] Nov 04 07:31:09.268292 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.268264 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:09.273354 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.273329 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6193165a-cd39-475b-b645-d16f3514cf62-config-file\") pod \"limitador-limitador-74dfcfb894-94g7x\" (UID: \"6193165a-cd39-475b-b645-d16f3514cf62\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" Nov 04 07:31:09.273465 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.273366 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn5w4\" (UniqueName: \"kubernetes.io/projected/6193165a-cd39-475b-b645-d16f3514cf62-kube-api-access-hn5w4\") pod \"limitador-limitador-74dfcfb894-94g7x\" (UID: \"6193165a-cd39-475b-b645-d16f3514cf62\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" Nov 04 07:31:09.367490 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.367459 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:09.374613 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.374588 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn5w4\" (UniqueName: \"kubernetes.io/projected/6193165a-cd39-475b-b645-d16f3514cf62-kube-api-access-hn5w4\") pod \"limitador-limitador-74dfcfb894-94g7x\" (UID: \"6193165a-cd39-475b-b645-d16f3514cf62\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" Nov 04 07:31:09.374733 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.374675 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6193165a-cd39-475b-b645-d16f3514cf62-config-file\") pod \"limitador-limitador-74dfcfb894-94g7x\" (UID: \"6193165a-cd39-475b-b645-d16f3514cf62\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" Nov 04 07:31:09.375270 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.375254 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6193165a-cd39-475b-b645-d16f3514cf62-config-file\") pod \"limitador-limitador-74dfcfb894-94g7x\" (UID: \"6193165a-cd39-475b-b645-d16f3514cf62\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" Nov 04 07:31:09.388065 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.388039 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn5w4\" (UniqueName: \"kubernetes.io/projected/6193165a-cd39-475b-b645-d16f3514cf62-kube-api-access-hn5w4\") pod \"limitador-limitador-74dfcfb894-94g7x\" (UID: \"6193165a-cd39-475b-b645-d16f3514cf62\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" Nov 04 07:31:09.473046 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.472979 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" Nov 04 07:31:09.744576 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.744477 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-94g7x"] Nov 04 07:31:09.747311 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:31:09.747276 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6193165a_cd39_475b_b645_d16f3514cf62.slice/crio-1f8f3ecf234e70ff2c35b9cb49cf49a5de3c3511505a263955026603b9483f9e WatchSource:0}: Error finding container 1f8f3ecf234e70ff2c35b9cb49cf49a5de3c3511505a263955026603b9483f9e: Status 404 returned error can't find the container with id 1f8f3ecf234e70ff2c35b9cb49cf49a5de3c3511505a263955026603b9483f9e Nov 04 07:31:09.749221 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.749205 5385 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 07:31:09.978383 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.978352 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-9kxk2"] Nov 04 07:31:09.982677 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.982657 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-9kxk2" Nov 04 07:31:09.989858 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.989836 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-9kxk2"] Nov 04 07:31:09.990162 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:09.990136 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c9nbb\"" Nov 04 07:31:10.051408 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:10.051335 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:10.079578 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:10.079546 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppw7c\" (UniqueName: \"kubernetes.io/projected/2cf37e6e-8d2a-4b13-863f-86e840d11d91-kube-api-access-ppw7c\") pod \"authorino-57f7f8659f-9kxk2\" (UID: \"2cf37e6e-8d2a-4b13-863f-86e840d11d91\") " pod="kuadrant-system/authorino-57f7f8659f-9kxk2" Nov 04 07:31:10.147081 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:10.147054 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:10.180158 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:10.180128 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppw7c\" (UniqueName: \"kubernetes.io/projected/2cf37e6e-8d2a-4b13-863f-86e840d11d91-kube-api-access-ppw7c\") pod \"authorino-57f7f8659f-9kxk2\" (UID: \"2cf37e6e-8d2a-4b13-863f-86e840d11d91\") " pod="kuadrant-system/authorino-57f7f8659f-9kxk2" Nov 04 07:31:10.192593 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:10.192569 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppw7c\" (UniqueName: \"kubernetes.io/projected/2cf37e6e-8d2a-4b13-863f-86e840d11d91-kube-api-access-ppw7c\") pod \"authorino-57f7f8659f-9kxk2\" (UID: \"2cf37e6e-8d2a-4b13-863f-86e840d11d91\") " pod="kuadrant-system/authorino-57f7f8659f-9kxk2" Nov 04 07:31:10.257099 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:10.257064 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" event={"ID":"6193165a-cd39-475b-b645-d16f3514cf62","Type":"ContainerStarted","Data":"1f8f3ecf234e70ff2c35b9cb49cf49a5de3c3511505a263955026603b9483f9e"} Nov 04 07:31:10.306564 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:10.306471 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-9kxk2" Nov 04 07:31:10.577979 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:10.577579 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-9kxk2"] Nov 04 07:31:10.581073 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:31:10.581033 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2cf37e6e_8d2a_4b13_863f_86e840d11d91.slice/crio-6142e550150dc3fa4f67952a4d55f8a7117188e7aca009e4bca2d5114d38ba4d WatchSource:0}: Error finding container 6142e550150dc3fa4f67952a4d55f8a7117188e7aca009e4bca2d5114d38ba4d: Status 404 returned error can't find the container with id 6142e550150dc3fa4f67952a4d55f8a7117188e7aca009e4bca2d5114d38ba4d Nov 04 07:31:11.263872 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:11.263797 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-9kxk2" event={"ID":"2cf37e6e-8d2a-4b13-863f-86e840d11d91","Type":"ContainerStarted","Data":"6142e550150dc3fa4f67952a4d55f8a7117188e7aca009e4bca2d5114d38ba4d"} Nov 04 07:31:13.277959 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:13.277876 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" event={"ID":"6193165a-cd39-475b-b645-d16f3514cf62","Type":"ContainerStarted","Data":"bde90d7c79fb7da3bd23340a6306f5380d28e04f68f2611eea279de3d87e3dd3"} Nov 04 07:31:13.278277 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:13.278105 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" Nov 04 07:31:13.298401 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:13.298335 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" podStartSLOduration=1.199705581 podStartE2EDuration="4.298316119s" totalImagesPullingTime="3.098610538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:31:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:31:13.297571413 +0000 UTC m=+908.631510997" watchObservedRunningTime="2026-11-04 07:31:13.298316119 +0000 UTC m=+908.632255707" Nov 04 07:31:14.238526 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:14.238477 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-9kxk2"] Nov 04 07:31:14.285894 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:14.285861 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-9kxk2" event={"ID":"2cf37e6e-8d2a-4b13-863f-86e840d11d91","Type":"ContainerStarted","Data":"a799377ae04f7eb34437f006a1965290316b9d7f11c526f0eaf5e584e1cf385f"} Nov 04 07:31:14.304002 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:14.303951 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-9kxk2" podStartSLOduration=2.653723575 podStartE2EDuration="5.303939048s" totalImagesPullingTime="2.650215473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:31:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:31:14.302111069 +0000 UTC m=+909.636050654" watchObservedRunningTime="2026-11-04 07:31:14.303939048 +0000 UTC m=+909.637878610" Nov 04 07:31:15.287981 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:15.287945 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-9kxk2" podUID="2cf37e6e-8d2a-4b13-863f-86e840d11d91" containerName="authorino" containerID="cri-o://a799377ae04f7eb34437f006a1965290316b9d7f11c526f0eaf5e584e1cf385f" gracePeriod=30 Nov 04 07:31:15.519286 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:15.519262 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-9kxk2" Nov 04 07:31:15.621837 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:15.621809 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2cf37e6e-8d2a-4b13-863f-86e840d11d91-kube-api-access-ppw7c\" (UniqueName: \"kubernetes.io/projected/2cf37e6e-8d2a-4b13-863f-86e840d11d91-kube-api-access-ppw7c\") pod \"2cf37e6e-8d2a-4b13-863f-86e840d11d91\" (UID: \"2cf37e6e-8d2a-4b13-863f-86e840d11d91\") " Nov 04 07:31:15.623929 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:15.623906 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cf37e6e-8d2a-4b13-863f-86e840d11d91-kube-api-access-ppw7c" pod "2cf37e6e-8d2a-4b13-863f-86e840d11d91" (UID: "2cf37e6e-8d2a-4b13-863f-86e840d11d91"). InnerVolumeSpecName "kube-api-access-ppw7c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:31:15.727194 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:15.722831 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppw7c\" (UniqueName: \"kubernetes.io/projected/2cf37e6e-8d2a-4b13-863f-86e840d11d91-kube-api-access-ppw7c\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:31:15.942789 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:15.942687 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:16.293168 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:16.293085 5385 generic.go:356] "Generic (PLEG): container finished" podID="2cf37e6e-8d2a-4b13-863f-86e840d11d91" containerID="a799377ae04f7eb34437f006a1965290316b9d7f11c526f0eaf5e584e1cf385f" exitCode=0 Nov 04 07:31:16.293168 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:16.293122 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-9kxk2" event={"ID":"2cf37e6e-8d2a-4b13-863f-86e840d11d91","Type":"ContainerDied","Data":"a799377ae04f7eb34437f006a1965290316b9d7f11c526f0eaf5e584e1cf385f"} Nov 04 07:31:16.293168 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:16.293125 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-9kxk2" Nov 04 07:31:16.293168 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:16.293148 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-9kxk2" event={"ID":"2cf37e6e-8d2a-4b13-863f-86e840d11d91","Type":"ContainerDied","Data":"6142e550150dc3fa4f67952a4d55f8a7117188e7aca009e4bca2d5114d38ba4d"} Nov 04 07:31:16.293168 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:16.293164 5385 scope.go:122] "RemoveContainer" containerID="a799377ae04f7eb34437f006a1965290316b9d7f11c526f0eaf5e584e1cf385f" Nov 04 07:31:16.301331 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:16.301305 5385 scope.go:122] "RemoveContainer" containerID="a799377ae04f7eb34437f006a1965290316b9d7f11c526f0eaf5e584e1cf385f" Nov 04 07:31:16.301626 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:31:16.301602 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a799377ae04f7eb34437f006a1965290316b9d7f11c526f0eaf5e584e1cf385f\": container with ID starting with a799377ae04f7eb34437f006a1965290316b9d7f11c526f0eaf5e584e1cf385f not found: ID does not exist" containerID="a799377ae04f7eb34437f006a1965290316b9d7f11c526f0eaf5e584e1cf385f" Nov 04 07:31:16.301707 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:16.301634 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a799377ae04f7eb34437f006a1965290316b9d7f11c526f0eaf5e584e1cf385f"} err="failed to get container status \"a799377ae04f7eb34437f006a1965290316b9d7f11c526f0eaf5e584e1cf385f\": rpc error: code = NotFound desc = could not find container \"a799377ae04f7eb34437f006a1965290316b9d7f11c526f0eaf5e584e1cf385f\": container with ID starting with a799377ae04f7eb34437f006a1965290316b9d7f11c526f0eaf5e584e1cf385f not found: ID does not exist" Nov 04 07:31:16.314103 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:16.314080 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-9kxk2"] Nov 04 07:31:16.318695 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:16.318674 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-9kxk2"] Nov 04 07:31:16.809697 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:16.809668 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2cf37e6e-8d2a-4b13-863f-86e840d11d91" path="/var/lib/kubelet/pods/2cf37e6e-8d2a-4b13-863f-86e840d11d91/volumes" Nov 04 07:31:24.286646 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:24.286615 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" Nov 04 07:31:27.873951 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:27.873913 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-94g7x"] Nov 04 07:31:27.874333 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:27.874099 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" podUID="6193165a-cd39-475b-b645-d16f3514cf62" containerName="limitador" containerID="cri-o://bde90d7c79fb7da3bd23340a6306f5380d28e04f68f2611eea279de3d87e3dd3" gracePeriod=30 Nov 04 07:31:28.340337 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:28.340306 5385 generic.go:356] "Generic (PLEG): container finished" podID="6193165a-cd39-475b-b645-d16f3514cf62" containerID="bde90d7c79fb7da3bd23340a6306f5380d28e04f68f2611eea279de3d87e3dd3" exitCode=0 Nov 04 07:31:28.340473 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:28.340346 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" event={"ID":"6193165a-cd39-475b-b645-d16f3514cf62","Type":"ContainerDied","Data":"bde90d7c79fb7da3bd23340a6306f5380d28e04f68f2611eea279de3d87e3dd3"} Nov 04 07:31:28.811224 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:28.811200 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" Nov 04 07:31:28.925524 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:28.925481 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6193165a-cd39-475b-b645-d16f3514cf62-kube-api-access-hn5w4\" (UniqueName: \"kubernetes.io/projected/6193165a-cd39-475b-b645-d16f3514cf62-kube-api-access-hn5w4\") pod \"6193165a-cd39-475b-b645-d16f3514cf62\" (UID: \"6193165a-cd39-475b-b645-d16f3514cf62\") " Nov 04 07:31:28.925936 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:28.925536 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6193165a-cd39-475b-b645-d16f3514cf62-config-file\" (UniqueName: \"kubernetes.io/configmap/6193165a-cd39-475b-b645-d16f3514cf62-config-file\") pod \"6193165a-cd39-475b-b645-d16f3514cf62\" (UID: \"6193165a-cd39-475b-b645-d16f3514cf62\") " Nov 04 07:31:28.925936 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:28.925916 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6193165a-cd39-475b-b645-d16f3514cf62-config-file" pod "6193165a-cd39-475b-b645-d16f3514cf62" (UID: "6193165a-cd39-475b-b645-d16f3514cf62"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:31:28.927673 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:28.927649 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6193165a-cd39-475b-b645-d16f3514cf62-kube-api-access-hn5w4" pod "6193165a-cd39-475b-b645-d16f3514cf62" (UID: "6193165a-cd39-475b-b645-d16f3514cf62"). InnerVolumeSpecName "kube-api-access-hn5w4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:31:29.026365 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:29.026290 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn5w4\" (UniqueName: \"kubernetes.io/projected/6193165a-cd39-475b-b645-d16f3514cf62-kube-api-access-hn5w4\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:31:29.026365 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:29.026318 5385 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6193165a-cd39-475b-b645-d16f3514cf62-config-file\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:31:29.345760 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:29.345672 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" event={"ID":"6193165a-cd39-475b-b645-d16f3514cf62","Type":"ContainerDied","Data":"1f8f3ecf234e70ff2c35b9cb49cf49a5de3c3511505a263955026603b9483f9e"} Nov 04 07:31:29.345760 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:29.345736 5385 scope.go:122] "RemoveContainer" containerID="bde90d7c79fb7da3bd23340a6306f5380d28e04f68f2611eea279de3d87e3dd3" Nov 04 07:31:29.345760 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:29.345684 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-94g7x" Nov 04 07:31:29.367596 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:29.367571 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-94g7x"] Nov 04 07:31:29.374110 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:29.374090 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-94g7x"] Nov 04 07:31:30.590897 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:30.590866 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:30.700145 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:30.700116 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:30.814900 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:30.814867 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6193165a-cd39-475b-b645-d16f3514cf62" path="/var/lib/kubelet/pods/6193165a-cd39-475b-b645-d16f3514cf62/volumes" Nov 04 07:31:30.873121 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:31:30.873091 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:31:31.316457 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.316378 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-n89nj"] Nov 04 07:31:31.316849 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.316827 5385 cpu_manager.go:414] "containerMap: removing container" podUID="6193165a-cd39-475b-b645-d16f3514cf62" containerName="limitador" Nov 04 07:31:31.316849 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.316848 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6193165a-cd39-475b-b645-d16f3514cf62" containerName="limitador" Nov 04 07:31:31.316992 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.316858 5385 cpu_manager.go:414] "containerMap: removing container" podUID="2cf37e6e-8d2a-4b13-863f-86e840d11d91" containerName="authorino" Nov 04 07:31:31.316992 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.316863 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2cf37e6e-8d2a-4b13-863f-86e840d11d91" containerName="authorino" Nov 04 07:31:31.316992 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.316936 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="6193165a-cd39-475b-b645-d16f3514cf62" containerName="limitador" Nov 04 07:31:31.316992 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.316945 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="2cf37e6e-8d2a-4b13-863f-86e840d11d91" containerName="authorino" Nov 04 07:31:31.321124 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.321107 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-n89nj" Nov 04 07:31:31.336190 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.336165 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Nov 04 07:31:31.338997 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.338978 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-n89nj"] Nov 04 07:31:31.344098 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.344081 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 07:31:31.344231 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.344214 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 07:31:31.413992 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.413967 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:31.436466 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.436443 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-tpq8g\"" Nov 04 07:31:31.444948 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.444930 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2b751fab-8c5e-4b42-bc7f-6a046157d11c-data\") pod \"postgres-7cdbf6dc44-n89nj\" (UID: \"2b751fab-8c5e-4b42-bc7f-6a046157d11c\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-n89nj" Nov 04 07:31:31.445011 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.444977 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvbcs\" (UniqueName: \"kubernetes.io/projected/2b751fab-8c5e-4b42-bc7f-6a046157d11c-kube-api-access-tvbcs\") pod \"postgres-7cdbf6dc44-n89nj\" (UID: \"2b751fab-8c5e-4b42-bc7f-6a046157d11c\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-n89nj" Nov 04 07:31:31.545885 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.545861 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2b751fab-8c5e-4b42-bc7f-6a046157d11c-data\") pod \"postgres-7cdbf6dc44-n89nj\" (UID: \"2b751fab-8c5e-4b42-bc7f-6a046157d11c\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-n89nj" Nov 04 07:31:31.545967 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.545904 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvbcs\" (UniqueName: \"kubernetes.io/projected/2b751fab-8c5e-4b42-bc7f-6a046157d11c-kube-api-access-tvbcs\") pod \"postgres-7cdbf6dc44-n89nj\" (UID: \"2b751fab-8c5e-4b42-bc7f-6a046157d11c\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-n89nj" Nov 04 07:31:31.546222 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.546206 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2b751fab-8c5e-4b42-bc7f-6a046157d11c-data\") pod \"postgres-7cdbf6dc44-n89nj\" (UID: \"2b751fab-8c5e-4b42-bc7f-6a046157d11c\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-n89nj" Nov 04 07:31:31.558651 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.558627 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvbcs\" (UniqueName: \"kubernetes.io/projected/2b751fab-8c5e-4b42-bc7f-6a046157d11c-kube-api-access-tvbcs\") pod \"postgres-7cdbf6dc44-n89nj\" (UID: \"2b751fab-8c5e-4b42-bc7f-6a046157d11c\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-n89nj" Nov 04 07:31:31.633166 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.633140 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-n89nj" Nov 04 07:31:31.901226 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:31.901155 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-n89nj"] Nov 04 07:31:31.904342 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:31:31.904315 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b751fab_8c5e_4b42_bc7f_6a046157d11c.slice/crio-50f8038061c7abbb45d4089601e2cd9642c07b7fa650c7e2a5df5184e6eb052f WatchSource:0}: Error finding container 50f8038061c7abbb45d4089601e2cd9642c07b7fa650c7e2a5df5184e6eb052f: Status 404 returned error can't find the container with id 50f8038061c7abbb45d4089601e2cd9642c07b7fa650c7e2a5df5184e6eb052f Nov 04 07:31:32.358960 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:32.358879 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-n89nj" event={"ID":"2b751fab-8c5e-4b42-bc7f-6a046157d11c","Type":"ContainerStarted","Data":"50f8038061c7abbb45d4089601e2cd9642c07b7fa650c7e2a5df5184e6eb052f"} Nov 04 07:31:37.387046 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:37.387008 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-n89nj" event={"ID":"2b751fab-8c5e-4b42-bc7f-6a046157d11c","Type":"ContainerStarted","Data":"50033d94c98aa1b625d32e68e3c06d1f6560f259e2ae0db25a8153722b6b7477"} Nov 04 07:31:37.387446 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:37.387093 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-n89nj" Nov 04 07:31:37.405955 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:37.405909 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-n89nj" podStartSLOduration=1.198358314 podStartE2EDuration="6.405895569s" totalImagesPullingTime="5.207537255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:31:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:31:37.405306003 +0000 UTC m=+932.739245588" watchObservedRunningTime="2026-11-04 07:31:37.405895569 +0000 UTC m=+932.739835153" Nov 04 07:31:38.420267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:38.420236 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:38.515804 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:38.515774 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:43.419582 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:43.419555 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-n89nj" Nov 04 07:31:50.323679 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.323639 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:50.402300 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.402267 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f5c75bcb9-lnh6s"] Nov 04 07:31:50.425195 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.425161 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f5c75bcb9-lnh6s"] Nov 04 07:31:50.425343 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.425258 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" Nov 04 07:31:50.428835 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.428792 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:50.430603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.430578 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Nov 04 07:31:50.430732 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.430670 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Nov 04 07:31:50.430732 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.430707 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Nov 04 07:31:50.430832 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.430750 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-6nx6g\"" Nov 04 07:31:50.495937 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.495911 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2l2c\" (UniqueName: \"kubernetes.io/projected/7d993f35-2749-4c4a-8d7b-da7d1f036b10-kube-api-access-g2l2c\") pod \"maas-controller-5f5c75bcb9-lnh6s\" (UID: \"7d993f35-2749-4c4a-8d7b-da7d1f036b10\") " pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" Nov 04 07:31:50.496075 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.495968 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d993f35-2749-4c4a-8d7b-da7d1f036b10-webhook-cert\") pod \"maas-controller-5f5c75bcb9-lnh6s\" (UID: \"7d993f35-2749-4c4a-8d7b-da7d1f036b10\") " pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" Nov 04 07:31:50.496127 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.496064 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d993f35-2749-4c4a-8d7b-da7d1f036b10-metrics-certs\") pod \"maas-controller-5f5c75bcb9-lnh6s\" (UID: \"7d993f35-2749-4c4a-8d7b-da7d1f036b10\") " pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" Nov 04 07:31:50.526159 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.526136 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:50.597379 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.597326 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2l2c\" (UniqueName: \"kubernetes.io/projected/7d993f35-2749-4c4a-8d7b-da7d1f036b10-kube-api-access-g2l2c\") pod \"maas-controller-5f5c75bcb9-lnh6s\" (UID: \"7d993f35-2749-4c4a-8d7b-da7d1f036b10\") " pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" Nov 04 07:31:50.597379 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.597375 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d993f35-2749-4c4a-8d7b-da7d1f036b10-webhook-cert\") pod \"maas-controller-5f5c75bcb9-lnh6s\" (UID: \"7d993f35-2749-4c4a-8d7b-da7d1f036b10\") " pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" Nov 04 07:31:50.597500 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.597405 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d993f35-2749-4c4a-8d7b-da7d1f036b10-metrics-certs\") pod \"maas-controller-5f5c75bcb9-lnh6s\" (UID: \"7d993f35-2749-4c4a-8d7b-da7d1f036b10\") " pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" Nov 04 07:31:50.599908 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.599887 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d993f35-2749-4c4a-8d7b-da7d1f036b10-metrics-certs\") pod \"maas-controller-5f5c75bcb9-lnh6s\" (UID: \"7d993f35-2749-4c4a-8d7b-da7d1f036b10\") " pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" Nov 04 07:31:50.599908 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.599898 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d993f35-2749-4c4a-8d7b-da7d1f036b10-webhook-cert\") pod \"maas-controller-5f5c75bcb9-lnh6s\" (UID: \"7d993f35-2749-4c4a-8d7b-da7d1f036b10\") " pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" Nov 04 07:31:50.609494 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.609473 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2l2c\" (UniqueName: \"kubernetes.io/projected/7d993f35-2749-4c4a-8d7b-da7d1f036b10-kube-api-access-g2l2c\") pod \"maas-controller-5f5c75bcb9-lnh6s\" (UID: \"7d993f35-2749-4c4a-8d7b-da7d1f036b10\") " pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" Nov 04 07:31:50.622461 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.622439 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:50.739061 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.739040 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" Nov 04 07:31:50.829043 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.829012 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-779b45f986-ng2mv"] Nov 04 07:31:50.832345 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.832323 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-779b45f986-ng2mv" Nov 04 07:31:50.843357 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.843332 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-779b45f986-ng2mv"] Nov 04 07:31:50.899861 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.899830 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c4eee75-8211-4d94-80de-20aeab2fbbac-metrics-certs\") pod \"maas-controller-779b45f986-ng2mv\" (UID: \"0c4eee75-8211-4d94-80de-20aeab2fbbac\") " pod="opendatahub/maas-controller-779b45f986-ng2mv" Nov 04 07:31:50.899985 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.899877 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gch6w\" (UniqueName: \"kubernetes.io/projected/0c4eee75-8211-4d94-80de-20aeab2fbbac-kube-api-access-gch6w\") pod \"maas-controller-779b45f986-ng2mv\" (UID: \"0c4eee75-8211-4d94-80de-20aeab2fbbac\") " pod="opendatahub/maas-controller-779b45f986-ng2mv" Nov 04 07:31:50.900029 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:50.900008 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c4eee75-8211-4d94-80de-20aeab2fbbac-webhook-cert\") pod \"maas-controller-779b45f986-ng2mv\" (UID: \"0c4eee75-8211-4d94-80de-20aeab2fbbac\") " pod="opendatahub/maas-controller-779b45f986-ng2mv" Nov 04 07:31:51.001180 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:51.001154 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gch6w\" (UniqueName: \"kubernetes.io/projected/0c4eee75-8211-4d94-80de-20aeab2fbbac-kube-api-access-gch6w\") pod \"maas-controller-779b45f986-ng2mv\" (UID: \"0c4eee75-8211-4d94-80de-20aeab2fbbac\") " pod="opendatahub/maas-controller-779b45f986-ng2mv" Nov 04 07:31:51.001312 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:51.001211 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c4eee75-8211-4d94-80de-20aeab2fbbac-webhook-cert\") pod \"maas-controller-779b45f986-ng2mv\" (UID: \"0c4eee75-8211-4d94-80de-20aeab2fbbac\") " pod="opendatahub/maas-controller-779b45f986-ng2mv" Nov 04 07:31:51.001312 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:51.001240 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c4eee75-8211-4d94-80de-20aeab2fbbac-metrics-certs\") pod \"maas-controller-779b45f986-ng2mv\" (UID: \"0c4eee75-8211-4d94-80de-20aeab2fbbac\") " pod="opendatahub/maas-controller-779b45f986-ng2mv" Nov 04 07:31:51.003539 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:51.003521 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0c4eee75-8211-4d94-80de-20aeab2fbbac-webhook-cert\") pod \"maas-controller-779b45f986-ng2mv\" (UID: \"0c4eee75-8211-4d94-80de-20aeab2fbbac\") " pod="opendatahub/maas-controller-779b45f986-ng2mv" Nov 04 07:31:51.003616 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:51.003559 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0c4eee75-8211-4d94-80de-20aeab2fbbac-metrics-certs\") pod \"maas-controller-779b45f986-ng2mv\" (UID: \"0c4eee75-8211-4d94-80de-20aeab2fbbac\") " pod="opendatahub/maas-controller-779b45f986-ng2mv" Nov 04 07:31:51.011931 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:51.011911 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f5c75bcb9-lnh6s"] Nov 04 07:31:51.013599 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:51.013583 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gch6w\" (UniqueName: \"kubernetes.io/projected/0c4eee75-8211-4d94-80de-20aeab2fbbac-kube-api-access-gch6w\") pod \"maas-controller-779b45f986-ng2mv\" (UID: \"0c4eee75-8211-4d94-80de-20aeab2fbbac\") " pod="opendatahub/maas-controller-779b45f986-ng2mv" Nov 04 07:31:51.015051 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:31:51.015022 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d993f35_2749_4c4a_8d7b_da7d1f036b10.slice/crio-ff483987f9d2dfca7c0da269b2f959136bfd610d46ca07602a78acb9b3b0004f WatchSource:0}: Error finding container ff483987f9d2dfca7c0da269b2f959136bfd610d46ca07602a78acb9b3b0004f: Status 404 returned error can't find the container with id ff483987f9d2dfca7c0da269b2f959136bfd610d46ca07602a78acb9b3b0004f Nov 04 07:31:51.144918 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:51.144893 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-779b45f986-ng2mv" Nov 04 07:31:51.419184 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:51.419118 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-779b45f986-ng2mv"] Nov 04 07:31:51.422278 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:31:51.422247 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c4eee75_8211_4d94_80de_20aeab2fbbac.slice/crio-28dc77427d6f992caa20047574b8f4b79351de12e9c2736d56d972495783e68e WatchSource:0}: Error finding container 28dc77427d6f992caa20047574b8f4b79351de12e9c2736d56d972495783e68e: Status 404 returned error can't find the container with id 28dc77427d6f992caa20047574b8f4b79351de12e9c2736d56d972495783e68e Nov 04 07:31:51.441820 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:51.441798 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-779b45f986-ng2mv" event={"ID":"0c4eee75-8211-4d94-80de-20aeab2fbbac","Type":"ContainerStarted","Data":"28dc77427d6f992caa20047574b8f4b79351de12e9c2736d56d972495783e68e"} Nov 04 07:31:51.442810 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:51.442788 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" event={"ID":"7d993f35-2749-4c4a-8d7b-da7d1f036b10","Type":"ContainerStarted","Data":"ff483987f9d2dfca7c0da269b2f959136bfd610d46ca07602a78acb9b3b0004f"} Nov 04 07:31:55.258857 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:55.258826 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:55.365399 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:55.365321 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:55.470618 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:55.470578 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-779b45f986-ng2mv" event={"ID":"0c4eee75-8211-4d94-80de-20aeab2fbbac","Type":"ContainerStarted","Data":"92872e8c2ee589f2975292e44f416e653610ae7364d08ebeab1742021ca942b1"} Nov 04 07:31:55.470784 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:55.470651 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-779b45f986-ng2mv" Nov 04 07:31:55.472047 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:55.472020 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" event={"ID":"7d993f35-2749-4c4a-8d7b-da7d1f036b10","Type":"ContainerStarted","Data":"95434834c89a3cdeaf672a999cdad4ed754508fac7d4c81561b6c98536340f4c"} Nov 04 07:31:55.472183 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:55.472165 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" Nov 04 07:31:55.494564 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:55.494484 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-779b45f986-ng2mv" podStartSLOduration=1.868712076 podStartE2EDuration="5.494469877s" totalImagesPullingTime="3.625757801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:31:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:31:55.492976961 +0000 UTC m=+950.826916546" watchObservedRunningTime="2026-11-04 07:31:55.494469877 +0000 UTC m=+950.828409462" Nov 04 07:31:55.513834 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:55.513782 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" podStartSLOduration=1.491957752 podStartE2EDuration="5.51376553s" totalImagesPullingTime="4.021807778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:31:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:31:55.512950433 +0000 UTC m=+950.846890019" watchObservedRunningTime="2026-11-04 07:31:55.51376553 +0000 UTC m=+950.847705115" Nov 04 07:31:56.673782 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.673749 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm"] Nov 04 07:31:56.677327 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.677303 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" Nov 04 07:31:56.683310 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.683283 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\"" Nov 04 07:31:56.683486 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.683294 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-dtwmf\"" Nov 04 07:31:56.683486 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.683315 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\"" Nov 04 07:31:56.683711 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.683333 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\"" Nov 04 07:31:56.688476 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.688452 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm"] Nov 04 07:31:56.743678 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.743625 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:56.852963 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.852928 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/44f8bc78-1413-4eba-a680-e22ff3b94f3b-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zgbmm\" (UID: \"44f8bc78-1413-4eba-a680-e22ff3b94f3b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" Nov 04 07:31:56.853139 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.852973 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/44f8bc78-1413-4eba-a680-e22ff3b94f3b-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zgbmm\" (UID: \"44f8bc78-1413-4eba-a680-e22ff3b94f3b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" Nov 04 07:31:56.853194 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.853141 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/44f8bc78-1413-4eba-a680-e22ff3b94f3b-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zgbmm\" (UID: \"44f8bc78-1413-4eba-a680-e22ff3b94f3b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" Nov 04 07:31:56.853194 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.853181 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b47s9\" (UniqueName: \"kubernetes.io/projected/44f8bc78-1413-4eba-a680-e22ff3b94f3b-kube-api-access-b47s9\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zgbmm\" (UID: \"44f8bc78-1413-4eba-a680-e22ff3b94f3b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" Nov 04 07:31:56.921592 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.921555 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:31:56.954148 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.954075 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/44f8bc78-1413-4eba-a680-e22ff3b94f3b-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zgbmm\" (UID: \"44f8bc78-1413-4eba-a680-e22ff3b94f3b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" Nov 04 07:31:56.954148 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.954129 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/44f8bc78-1413-4eba-a680-e22ff3b94f3b-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zgbmm\" (UID: \"44f8bc78-1413-4eba-a680-e22ff3b94f3b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" Nov 04 07:31:56.954369 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.954149 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b47s9\" (UniqueName: \"kubernetes.io/projected/44f8bc78-1413-4eba-a680-e22ff3b94f3b-kube-api-access-b47s9\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zgbmm\" (UID: \"44f8bc78-1413-4eba-a680-e22ff3b94f3b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" Nov 04 07:31:56.954369 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.954190 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/44f8bc78-1413-4eba-a680-e22ff3b94f3b-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zgbmm\" (UID: \"44f8bc78-1413-4eba-a680-e22ff3b94f3b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" Nov 04 07:31:56.954845 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.954820 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/44f8bc78-1413-4eba-a680-e22ff3b94f3b-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zgbmm\" (UID: \"44f8bc78-1413-4eba-a680-e22ff3b94f3b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" Nov 04 07:31:56.955058 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.955040 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/44f8bc78-1413-4eba-a680-e22ff3b94f3b-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zgbmm\" (UID: \"44f8bc78-1413-4eba-a680-e22ff3b94f3b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" Nov 04 07:31:56.956588 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.956558 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/44f8bc78-1413-4eba-a680-e22ff3b94f3b-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zgbmm\" (UID: \"44f8bc78-1413-4eba-a680-e22ff3b94f3b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" Nov 04 07:31:56.966817 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.966794 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b47s9\" (UniqueName: \"kubernetes.io/projected/44f8bc78-1413-4eba-a680-e22ff3b94f3b-kube-api-access-b47s9\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-zgbmm\" (UID: \"44f8bc78-1413-4eba-a680-e22ff3b94f3b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" Nov 04 07:31:56.993232 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:56.993208 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" Nov 04 07:31:57.268403 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:57.268329 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm"] Nov 04 07:31:57.271359 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:31:57.271332 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44f8bc78_1413_4eba_a680_e22ff3b94f3b.slice/crio-0d38d2ede2aa40bd1637c3580e7dd54e43828a15512b2cf320e69c9883255cee WatchSource:0}: Error finding container 0d38d2ede2aa40bd1637c3580e7dd54e43828a15512b2cf320e69c9883255cee: Status 404 returned error can't find the container with id 0d38d2ede2aa40bd1637c3580e7dd54e43828a15512b2cf320e69c9883255cee Nov 04 07:31:57.481856 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:31:57.481822 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" event={"ID":"44f8bc78-1413-4eba-a680-e22ff3b94f3b","Type":"ContainerStarted","Data":"0d38d2ede2aa40bd1637c3580e7dd54e43828a15512b2cf320e69c9883255cee"} Nov 04 07:32:00.874088 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:32:00.874046 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:32:06.480098 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:06.480063 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-779b45f986-ng2mv" Nov 04 07:32:06.480615 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:06.480138 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" Nov 04 07:32:06.539532 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:06.539470 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f5c75bcb9-lnh6s"] Nov 04 07:32:06.539767 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:06.539737 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" podUID="7d993f35-2749-4c4a-8d7b-da7d1f036b10" containerName="manager" containerID="cri-o://95434834c89a3cdeaf672a999cdad4ed754508fac7d4c81561b6c98536340f4c" gracePeriod=30 Nov 04 07:32:06.828840 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:06.825840 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" Nov 04 07:32:06.843871 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:06.843837 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7d993f35-2749-4c4a-8d7b-da7d1f036b10-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d993f35-2749-4c4a-8d7b-da7d1f036b10-metrics-certs\") pod \"7d993f35-2749-4c4a-8d7b-da7d1f036b10\" (UID: \"7d993f35-2749-4c4a-8d7b-da7d1f036b10\") " Nov 04 07:32:06.844009 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:06.843913 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d993f35-2749-4c4a-8d7b-da7d1f036b10-kube-api-access-g2l2c\" (UniqueName: \"kubernetes.io/projected/7d993f35-2749-4c4a-8d7b-da7d1f036b10-kube-api-access-g2l2c\") pod \"7d993f35-2749-4c4a-8d7b-da7d1f036b10\" (UID: \"7d993f35-2749-4c4a-8d7b-da7d1f036b10\") " Nov 04 07:32:06.844076 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:06.844015 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7d993f35-2749-4c4a-8d7b-da7d1f036b10-webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d993f35-2749-4c4a-8d7b-da7d1f036b10-webhook-cert\") pod \"7d993f35-2749-4c4a-8d7b-da7d1f036b10\" (UID: \"7d993f35-2749-4c4a-8d7b-da7d1f036b10\") " Nov 04 07:32:06.848408 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:06.848371 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d993f35-2749-4c4a-8d7b-da7d1f036b10-kube-api-access-g2l2c" pod "7d993f35-2749-4c4a-8d7b-da7d1f036b10" (UID: "7d993f35-2749-4c4a-8d7b-da7d1f036b10"). InnerVolumeSpecName "kube-api-access-g2l2c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:32:06.850536 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:06.850473 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d993f35-2749-4c4a-8d7b-da7d1f036b10-metrics-certs" pod "7d993f35-2749-4c4a-8d7b-da7d1f036b10" (UID: "7d993f35-2749-4c4a-8d7b-da7d1f036b10"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:32:06.851706 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:06.851668 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d993f35-2749-4c4a-8d7b-da7d1f036b10-webhook-cert" pod "7d993f35-2749-4c4a-8d7b-da7d1f036b10" (UID: "7d993f35-2749-4c4a-8d7b-da7d1f036b10"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:32:06.945218 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:06.945178 5385 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7d993f35-2749-4c4a-8d7b-da7d1f036b10-webhook-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:32:06.945218 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:06.945217 5385 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d993f35-2749-4c4a-8d7b-da7d1f036b10-metrics-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:32:06.945426 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:06.945229 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2l2c\" (UniqueName: \"kubernetes.io/projected/7d993f35-2749-4c4a-8d7b-da7d1f036b10-kube-api-access-g2l2c\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:32:07.528934 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:07.528839 5385 generic.go:356] "Generic (PLEG): container finished" podID="7d993f35-2749-4c4a-8d7b-da7d1f036b10" containerID="95434834c89a3cdeaf672a999cdad4ed754508fac7d4c81561b6c98536340f4c" exitCode=0 Nov 04 07:32:07.528934 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:07.528899 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" Nov 04 07:32:07.529485 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:07.528889 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" event={"ID":"7d993f35-2749-4c4a-8d7b-da7d1f036b10","Type":"ContainerDied","Data":"95434834c89a3cdeaf672a999cdad4ed754508fac7d4c81561b6c98536340f4c"} Nov 04 07:32:07.529485 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:07.529029 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5c75bcb9-lnh6s" event={"ID":"7d993f35-2749-4c4a-8d7b-da7d1f036b10","Type":"ContainerDied","Data":"ff483987f9d2dfca7c0da269b2f959136bfd610d46ca07602a78acb9b3b0004f"} Nov 04 07:32:07.529485 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:07.529063 5385 scope.go:122] "RemoveContainer" containerID="95434834c89a3cdeaf672a999cdad4ed754508fac7d4c81561b6c98536340f4c" Nov 04 07:32:07.547374 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:07.547346 5385 scope.go:122] "RemoveContainer" containerID="95434834c89a3cdeaf672a999cdad4ed754508fac7d4c81561b6c98536340f4c" Nov 04 07:32:07.547789 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:32:07.547762 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95434834c89a3cdeaf672a999cdad4ed754508fac7d4c81561b6c98536340f4c\": container with ID starting with 95434834c89a3cdeaf672a999cdad4ed754508fac7d4c81561b6c98536340f4c not found: ID does not exist" containerID="95434834c89a3cdeaf672a999cdad4ed754508fac7d4c81561b6c98536340f4c" Nov 04 07:32:07.547879 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:07.547804 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95434834c89a3cdeaf672a999cdad4ed754508fac7d4c81561b6c98536340f4c"} err="failed to get container status \"95434834c89a3cdeaf672a999cdad4ed754508fac7d4c81561b6c98536340f4c\": rpc error: code = NotFound desc = could not find container \"95434834c89a3cdeaf672a999cdad4ed754508fac7d4c81561b6c98536340f4c\": container with ID starting with 95434834c89a3cdeaf672a999cdad4ed754508fac7d4c81561b6c98536340f4c not found: ID does not exist" Nov 04 07:32:07.555984 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:07.555958 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f5c75bcb9-lnh6s"] Nov 04 07:32:07.561940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:07.561914 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f5c75bcb9-lnh6s"] Nov 04 07:32:08.810479 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:08.810446 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d993f35-2749-4c4a-8d7b-da7d1f036b10" path="/var/lib/kubelet/pods/7d993f35-2749-4c4a-8d7b-da7d1f036b10/volumes" Nov 04 07:32:14.561917 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:14.561879 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" event={"ID":"44f8bc78-1413-4eba-a680-e22ff3b94f3b","Type":"ContainerStarted","Data":"af8ecd2ed52cee22c3645aa80164b9c9473e65dac2b347c1c157ba3acb9054d9"} Nov 04 07:32:14.562371 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:14.562027 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" Nov 04 07:32:14.582045 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:14.581998 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" podStartSLOduration=2.324083516 podStartE2EDuration="18.581986071s" totalImagesPullingTime="16.257902555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:31:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:32:14.580812491 +0000 UTC m=+969.914752075" watchObservedRunningTime="2026-11-04 07:32:14.581986071 +0000 UTC m=+969.915925655" Nov 04 07:32:16.003699 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:16.003663 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:32:25.571586 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:25.571550 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zgbmm" Nov 04 07:32:27.072053 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:27.072011 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:32:27.211189 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:27.211160 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:32:28.454366 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.454328 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:32:28.565720 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.565687 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:32:28.602576 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.602546 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2"] Nov 04 07:32:28.602895 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.602863 5385 cpu_manager.go:414] "containerMap: removing container" podUID="7d993f35-2749-4c4a-8d7b-da7d1f036b10" containerName="manager" Nov 04 07:32:28.602895 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.602876 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d993f35-2749-4c4a-8d7b-da7d1f036b10" containerName="manager" Nov 04 07:32:28.602980 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.602944 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d993f35-2749-4c4a-8d7b-da7d1f036b10" containerName="manager" Nov 04 07:32:28.608690 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.608668 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:28.614165 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.614146 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hv4qq\"" Nov 04 07:32:28.619584 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.619562 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2"] Nov 04 07:32:28.684318 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.684289 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:32:28.715575 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.715513 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Nov 04 07:32:28.715575 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.715522 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Nov 04 07:32:28.716809 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.716789 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/222fb595-75f2-40c6-ba72-fcae616e3686-openshift-service-ca\") pod \"maas-api-5bcd5677d6-5xwm2\" (UID: \"222fb595-75f2-40c6-ba72-fcae616e3686\") " pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:28.716862 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.716838 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvxb2\" (UniqueName: \"kubernetes.io/projected/222fb595-75f2-40c6-ba72-fcae616e3686-kube-api-access-qvxb2\") pod \"maas-api-5bcd5677d6-5xwm2\" (UID: \"222fb595-75f2-40c6-ba72-fcae616e3686\") " pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:28.716862 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.716857 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/222fb595-75f2-40c6-ba72-fcae616e3686-maas-api-tls\") pod \"maas-api-5bcd5677d6-5xwm2\" (UID: \"222fb595-75f2-40c6-ba72-fcae616e3686\") " pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:28.716952 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.716919 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222fb595-75f2-40c6-ba72-fcae616e3686-metrics-certs\") pod \"maas-api-5bcd5677d6-5xwm2\" (UID: \"222fb595-75f2-40c6-ba72-fcae616e3686\") " pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:28.809213 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.809127 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:32:28.818120 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.818092 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/222fb595-75f2-40c6-ba72-fcae616e3686-openshift-service-ca\") pod \"maas-api-5bcd5677d6-5xwm2\" (UID: \"222fb595-75f2-40c6-ba72-fcae616e3686\") " pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:28.818261 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.818141 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvxb2\" (UniqueName: \"kubernetes.io/projected/222fb595-75f2-40c6-ba72-fcae616e3686-kube-api-access-qvxb2\") pod \"maas-api-5bcd5677d6-5xwm2\" (UID: \"222fb595-75f2-40c6-ba72-fcae616e3686\") " pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:28.818261 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.818163 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/222fb595-75f2-40c6-ba72-fcae616e3686-maas-api-tls\") pod \"maas-api-5bcd5677d6-5xwm2\" (UID: \"222fb595-75f2-40c6-ba72-fcae616e3686\") " pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:28.818377 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.818276 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222fb595-75f2-40c6-ba72-fcae616e3686-metrics-certs\") pod \"maas-api-5bcd5677d6-5xwm2\" (UID: \"222fb595-75f2-40c6-ba72-fcae616e3686\") " pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:28.818928 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.818905 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/222fb595-75f2-40c6-ba72-fcae616e3686-openshift-service-ca\") pod \"maas-api-5bcd5677d6-5xwm2\" (UID: \"222fb595-75f2-40c6-ba72-fcae616e3686\") " pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:28.820889 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.820866 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/222fb595-75f2-40c6-ba72-fcae616e3686-maas-api-tls\") pod \"maas-api-5bcd5677d6-5xwm2\" (UID: \"222fb595-75f2-40c6-ba72-fcae616e3686\") " pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:28.821113 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.821094 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222fb595-75f2-40c6-ba72-fcae616e3686-metrics-certs\") pod \"maas-api-5bcd5677d6-5xwm2\" (UID: \"222fb595-75f2-40c6-ba72-fcae616e3686\") " pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:28.830083 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.830065 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvxb2\" (UniqueName: \"kubernetes.io/projected/222fb595-75f2-40c6-ba72-fcae616e3686-kube-api-access-qvxb2\") pod \"maas-api-5bcd5677d6-5xwm2\" (UID: \"222fb595-75f2-40c6-ba72-fcae616e3686\") " pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:28.922547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:28.922489 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:29.201341 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:29.201176 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2"] Nov 04 07:32:29.204051 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:32:29.204024 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod222fb595_75f2_40c6_ba72_fcae616e3686.slice/crio-e0a3c356bd7a8a28b5316311ae260fe51792d37d03a31d1d67b65ce0577f667a WatchSource:0}: Error finding container e0a3c356bd7a8a28b5316311ae260fe51792d37d03a31d1d67b65ce0577f667a: Status 404 returned error can't find the container with id e0a3c356bd7a8a28b5316311ae260fe51792d37d03a31d1d67b65ce0577f667a Nov 04 07:32:29.623611 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:29.623575 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" event={"ID":"222fb595-75f2-40c6-ba72-fcae616e3686","Type":"ContainerStarted","Data":"e0a3c356bd7a8a28b5316311ae260fe51792d37d03a31d1d67b65ce0577f667a"} Nov 04 07:32:30.874732 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:32:30.874699 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:32:33.648114 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:33.648076 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" event={"ID":"222fb595-75f2-40c6-ba72-fcae616e3686","Type":"ContainerStarted","Data":"95bb83c8b9af668cb510d6ddd0549770461edb8286d4bfa77aec1089dda7fba7"} Nov 04 07:32:33.648522 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:33.648211 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:33.669631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:33.669576 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" podStartSLOduration=2.283896329 podStartE2EDuration="5.669562808s" totalImagesPullingTime="3.385666479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:32:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:32:33.669068321 +0000 UTC m=+989.003007901" watchObservedRunningTime="2026-11-04 07:32:33.669562808 +0000 UTC m=+989.003502393" Nov 04 07:32:39.658855 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:39.658824 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:40.083578 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.083485 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:32:40.183518 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.183482 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:32:40.361973 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.361900 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6dd76875b5-cdbfb"] Nov 04 07:32:40.378816 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.378786 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dd76875b5-cdbfb"] Nov 04 07:32:40.378999 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.378876 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd76875b5-cdbfb" Nov 04 07:32:40.382703 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.382678 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Nov 04 07:32:40.382808 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.382769 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Nov 04 07:32:40.383985 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.383967 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c9nbb\"" Nov 04 07:32:40.384085 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.384051 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Nov 04 07:32:40.395318 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.395287 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dd76875b5-cdbfb"] Nov 04 07:32:40.403877 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:32:40.403855 5385 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rl8l5 tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-rl8l5 tls-cert]: context canceled" pod="kuadrant-system/authorino-6dd76875b5-cdbfb" podUID="67312c83-cc63-405d-ada6-e130a7fe0b30" Nov 04 07:32:40.513580 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.513542 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rl8l5\" (UniqueName: \"kubernetes.io/projected/67312c83-cc63-405d-ada6-e130a7fe0b30-kube-api-access-rl8l5\") pod \"authorino-6dd76875b5-cdbfb\" (UID: \"67312c83-cc63-405d-ada6-e130a7fe0b30\") " pod="kuadrant-system/authorino-6dd76875b5-cdbfb" Nov 04 07:32:40.513729 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.513680 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/67312c83-cc63-405d-ada6-e130a7fe0b30-tls-cert\") pod \"authorino-6dd76875b5-cdbfb\" (UID: \"67312c83-cc63-405d-ada6-e130a7fe0b30\") " pod="kuadrant-system/authorino-6dd76875b5-cdbfb" Nov 04 07:32:40.614225 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.614136 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rl8l5\" (UniqueName: \"kubernetes.io/projected/67312c83-cc63-405d-ada6-e130a7fe0b30-kube-api-access-rl8l5\") pod \"authorino-6dd76875b5-cdbfb\" (UID: \"67312c83-cc63-405d-ada6-e130a7fe0b30\") " pod="kuadrant-system/authorino-6dd76875b5-cdbfb" Nov 04 07:32:40.614365 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.614222 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/67312c83-cc63-405d-ada6-e130a7fe0b30-tls-cert\") pod \"authorino-6dd76875b5-cdbfb\" (UID: \"67312c83-cc63-405d-ada6-e130a7fe0b30\") " pod="kuadrant-system/authorino-6dd76875b5-cdbfb" Nov 04 07:32:40.616591 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.616564 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/67312c83-cc63-405d-ada6-e130a7fe0b30-tls-cert\") pod \"authorino-6dd76875b5-cdbfb\" (UID: \"67312c83-cc63-405d-ada6-e130a7fe0b30\") " pod="kuadrant-system/authorino-6dd76875b5-cdbfb" Nov 04 07:32:40.626804 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.626774 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rl8l5\" (UniqueName: \"kubernetes.io/projected/67312c83-cc63-405d-ada6-e130a7fe0b30-kube-api-access-rl8l5\") pod \"authorino-6dd76875b5-cdbfb\" (UID: \"67312c83-cc63-405d-ada6-e130a7fe0b30\") " pod="kuadrant-system/authorino-6dd76875b5-cdbfb" Nov 04 07:32:40.677006 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.676977 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd76875b5-cdbfb" Nov 04 07:32:40.684035 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.684010 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd76875b5-cdbfb" Nov 04 07:32:40.742435 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.742402 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5ffbd6df97-qw2pm"] Nov 04 07:32:40.745682 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.745657 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ffbd6df97-qw2pm" Nov 04 07:32:40.754345 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.754321 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5ffbd6df97-qw2pm"] Nov 04 07:32:40.816587 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.816554 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67312c83-cc63-405d-ada6-e130a7fe0b30-kube-api-access-rl8l5\" (UniqueName: \"kubernetes.io/projected/67312c83-cc63-405d-ada6-e130a7fe0b30-kube-api-access-rl8l5\") pod \"67312c83-cc63-405d-ada6-e130a7fe0b30\" (UID: \"67312c83-cc63-405d-ada6-e130a7fe0b30\") " Nov 04 07:32:40.816736 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.816603 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/67312c83-cc63-405d-ada6-e130a7fe0b30-tls-cert\" (UniqueName: \"kubernetes.io/secret/67312c83-cc63-405d-ada6-e130a7fe0b30-tls-cert\") pod \"67312c83-cc63-405d-ada6-e130a7fe0b30\" (UID: \"67312c83-cc63-405d-ada6-e130a7fe0b30\") " Nov 04 07:32:40.816736 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.816699 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2fc5119a-070d-4016-8124-ba6a546d9e35-tls-cert\") pod \"authorino-5ffbd6df97-qw2pm\" (UID: \"2fc5119a-070d-4016-8124-ba6a546d9e35\") " pod="kuadrant-system/authorino-5ffbd6df97-qw2pm" Nov 04 07:32:40.816804 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.816741 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5pf9\" (UniqueName: \"kubernetes.io/projected/2fc5119a-070d-4016-8124-ba6a546d9e35-kube-api-access-r5pf9\") pod \"authorino-5ffbd6df97-qw2pm\" (UID: \"2fc5119a-070d-4016-8124-ba6a546d9e35\") " pod="kuadrant-system/authorino-5ffbd6df97-qw2pm" Nov 04 07:32:40.818551 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.818519 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67312c83-cc63-405d-ada6-e130a7fe0b30-kube-api-access-rl8l5" pod "67312c83-cc63-405d-ada6-e130a7fe0b30" (UID: "67312c83-cc63-405d-ada6-e130a7fe0b30"). InnerVolumeSpecName "kube-api-access-rl8l5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:32:40.818551 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.818541 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67312c83-cc63-405d-ada6-e130a7fe0b30-tls-cert" pod "67312c83-cc63-405d-ada6-e130a7fe0b30" (UID: "67312c83-cc63-405d-ada6-e130a7fe0b30"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:32:40.918173 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.918142 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5pf9\" (UniqueName: \"kubernetes.io/projected/2fc5119a-070d-4016-8124-ba6a546d9e35-kube-api-access-r5pf9\") pod \"authorino-5ffbd6df97-qw2pm\" (UID: \"2fc5119a-070d-4016-8124-ba6a546d9e35\") " pod="kuadrant-system/authorino-5ffbd6df97-qw2pm" Nov 04 07:32:40.918315 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.918200 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2fc5119a-070d-4016-8124-ba6a546d9e35-tls-cert\") pod \"authorino-5ffbd6df97-qw2pm\" (UID: \"2fc5119a-070d-4016-8124-ba6a546d9e35\") " pod="kuadrant-system/authorino-5ffbd6df97-qw2pm" Nov 04 07:32:40.918315 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.918230 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rl8l5\" (UniqueName: \"kubernetes.io/projected/67312c83-cc63-405d-ada6-e130a7fe0b30-kube-api-access-rl8l5\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:32:40.918315 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.918240 5385 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/67312c83-cc63-405d-ada6-e130a7fe0b30-tls-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:32:40.920572 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.920548 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2fc5119a-070d-4016-8124-ba6a546d9e35-tls-cert\") pod \"authorino-5ffbd6df97-qw2pm\" (UID: \"2fc5119a-070d-4016-8124-ba6a546d9e35\") " pod="kuadrant-system/authorino-5ffbd6df97-qw2pm" Nov 04 07:32:40.930736 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:40.930712 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5pf9\" (UniqueName: \"kubernetes.io/projected/2fc5119a-070d-4016-8124-ba6a546d9e35-kube-api-access-r5pf9\") pod \"authorino-5ffbd6df97-qw2pm\" (UID: \"2fc5119a-070d-4016-8124-ba6a546d9e35\") " pod="kuadrant-system/authorino-5ffbd6df97-qw2pm" Nov 04 07:32:41.057521 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:41.057478 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ffbd6df97-qw2pm" Nov 04 07:32:41.328121 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:41.327933 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5ffbd6df97-qw2pm"] Nov 04 07:32:41.330581 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:32:41.330558 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fc5119a_070d_4016_8124_ba6a546d9e35.slice/crio-b8502d2775270ae49a44c6125b496d990ada46446137f4236bdf3ae7a60a5066 WatchSource:0}: Error finding container b8502d2775270ae49a44c6125b496d990ada46446137f4236bdf3ae7a60a5066: Status 404 returned error can't find the container with id b8502d2775270ae49a44c6125b496d990ada46446137f4236bdf3ae7a60a5066 Nov 04 07:32:41.682281 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:41.682244 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ffbd6df97-qw2pm" event={"ID":"2fc5119a-070d-4016-8124-ba6a546d9e35","Type":"ContainerStarted","Data":"b8502d2775270ae49a44c6125b496d990ada46446137f4236bdf3ae7a60a5066"} Nov 04 07:32:41.682281 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:41.682279 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd76875b5-cdbfb" Nov 04 07:32:41.717006 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:41.716973 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dd76875b5-cdbfb"] Nov 04 07:32:41.720516 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:41.720485 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6dd76875b5-cdbfb"] Nov 04 07:32:42.687982 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:42.687940 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ffbd6df97-qw2pm" event={"ID":"2fc5119a-070d-4016-8124-ba6a546d9e35","Type":"ContainerStarted","Data":"fe3bdcc0cb7600310d01ff79c98f5dbc05e39279940441639df123f519dbf047"} Nov 04 07:32:42.709202 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:42.709145 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-5ffbd6df97-qw2pm" podStartSLOduration=2.025483212 podStartE2EDuration="2.709131075s" totalImagesPullingTime="683.647863ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:32:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:32:42.708015808 +0000 UTC m=+998.041955392" watchObservedRunningTime="2026-11-04 07:32:42.709131075 +0000 UTC m=+998.043070659" Nov 04 07:32:42.811425 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:42.811392 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="67312c83-cc63-405d-ada6-e130a7fe0b30" path="/var/lib/kubelet/pods/67312c83-cc63-405d-ada6-e130a7fe0b30/volumes" Nov 04 07:32:50.328687 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.328611 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2"] Nov 04 07:32:50.329256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.328840 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" podUID="222fb595-75f2-40c6-ba72-fcae616e3686" containerName="maas-api" containerID="cri-o://95bb83c8b9af668cb510d6ddd0549770461edb8286d4bfa77aec1089dda7fba7" gracePeriod=30 Nov 04 07:32:50.567993 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.567970 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:50.703877 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.703840 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/222fb595-75f2-40c6-ba72-fcae616e3686-metrics-certs\" (UniqueName: \"kubernetes.io/secret/222fb595-75f2-40c6-ba72-fcae616e3686-metrics-certs\") pod \"222fb595-75f2-40c6-ba72-fcae616e3686\" (UID: \"222fb595-75f2-40c6-ba72-fcae616e3686\") " Nov 04 07:32:50.704058 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.703886 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/222fb595-75f2-40c6-ba72-fcae616e3686-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/222fb595-75f2-40c6-ba72-fcae616e3686-openshift-service-ca\") pod \"222fb595-75f2-40c6-ba72-fcae616e3686\" (UID: \"222fb595-75f2-40c6-ba72-fcae616e3686\") " Nov 04 07:32:50.704058 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.703924 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/222fb595-75f2-40c6-ba72-fcae616e3686-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/222fb595-75f2-40c6-ba72-fcae616e3686-maas-api-tls\") pod \"222fb595-75f2-40c6-ba72-fcae616e3686\" (UID: \"222fb595-75f2-40c6-ba72-fcae616e3686\") " Nov 04 07:32:50.704058 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.704040 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/222fb595-75f2-40c6-ba72-fcae616e3686-kube-api-access-qvxb2\" (UniqueName: \"kubernetes.io/projected/222fb595-75f2-40c6-ba72-fcae616e3686-kube-api-access-qvxb2\") pod \"222fb595-75f2-40c6-ba72-fcae616e3686\" (UID: \"222fb595-75f2-40c6-ba72-fcae616e3686\") " Nov 04 07:32:50.704296 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.704273 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/222fb595-75f2-40c6-ba72-fcae616e3686-openshift-service-ca" pod "222fb595-75f2-40c6-ba72-fcae616e3686" (UID: "222fb595-75f2-40c6-ba72-fcae616e3686"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:32:50.706095 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.706070 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/222fb595-75f2-40c6-ba72-fcae616e3686-metrics-certs" pod "222fb595-75f2-40c6-ba72-fcae616e3686" (UID: "222fb595-75f2-40c6-ba72-fcae616e3686"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:32:50.706189 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.706123 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/222fb595-75f2-40c6-ba72-fcae616e3686-maas-api-tls" pod "222fb595-75f2-40c6-ba72-fcae616e3686" (UID: "222fb595-75f2-40c6-ba72-fcae616e3686"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:32:50.706189 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.706170 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/222fb595-75f2-40c6-ba72-fcae616e3686-kube-api-access-qvxb2" pod "222fb595-75f2-40c6-ba72-fcae616e3686" (UID: "222fb595-75f2-40c6-ba72-fcae616e3686"). InnerVolumeSpecName "kube-api-access-qvxb2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:32:50.722316 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.722287 5385 generic.go:356] "Generic (PLEG): container finished" podID="222fb595-75f2-40c6-ba72-fcae616e3686" containerID="95bb83c8b9af668cb510d6ddd0549770461edb8286d4bfa77aec1089dda7fba7" exitCode=0 Nov 04 07:32:50.722417 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.722334 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" Nov 04 07:32:50.722417 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.722363 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" event={"ID":"222fb595-75f2-40c6-ba72-fcae616e3686","Type":"ContainerDied","Data":"95bb83c8b9af668cb510d6ddd0549770461edb8286d4bfa77aec1089dda7fba7"} Nov 04 07:32:50.722490 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.722431 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2" event={"ID":"222fb595-75f2-40c6-ba72-fcae616e3686","Type":"ContainerDied","Data":"e0a3c356bd7a8a28b5316311ae260fe51792d37d03a31d1d67b65ce0577f667a"} Nov 04 07:32:50.722490 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.722457 5385 scope.go:122] "RemoveContainer" containerID="95bb83c8b9af668cb510d6ddd0549770461edb8286d4bfa77aec1089dda7fba7" Nov 04 07:32:50.732743 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.732723 5385 scope.go:122] "RemoveContainer" containerID="95bb83c8b9af668cb510d6ddd0549770461edb8286d4bfa77aec1089dda7fba7" Nov 04 07:32:50.732984 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:32:50.732963 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95bb83c8b9af668cb510d6ddd0549770461edb8286d4bfa77aec1089dda7fba7\": container with ID starting with 95bb83c8b9af668cb510d6ddd0549770461edb8286d4bfa77aec1089dda7fba7 not found: ID does not exist" containerID="95bb83c8b9af668cb510d6ddd0549770461edb8286d4bfa77aec1089dda7fba7" Nov 04 07:32:50.733030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.732998 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95bb83c8b9af668cb510d6ddd0549770461edb8286d4bfa77aec1089dda7fba7"} err="failed to get container status \"95bb83c8b9af668cb510d6ddd0549770461edb8286d4bfa77aec1089dda7fba7\": rpc error: code = NotFound desc = could not find container \"95bb83c8b9af668cb510d6ddd0549770461edb8286d4bfa77aec1089dda7fba7\": container with ID starting with 95bb83c8b9af668cb510d6ddd0549770461edb8286d4bfa77aec1089dda7fba7 not found: ID does not exist" Nov 04 07:32:50.743883 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.743860 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2"] Nov 04 07:32:50.751975 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.751956 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5bcd5677d6-5xwm2"] Nov 04 07:32:50.804852 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.804822 5385 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/222fb595-75f2-40c6-ba72-fcae616e3686-metrics-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:32:50.804852 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.804850 5385 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/222fb595-75f2-40c6-ba72-fcae616e3686-openshift-service-ca\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:32:50.805023 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.804862 5385 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/222fb595-75f2-40c6-ba72-fcae616e3686-maas-api-tls\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:32:50.805023 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.804871 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvxb2\" (UniqueName: \"kubernetes.io/projected/222fb595-75f2-40c6-ba72-fcae616e3686-kube-api-access-qvxb2\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:32:50.810367 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:32:50.810340 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="222fb595-75f2-40c6-ba72-fcae616e3686" path="/var/lib/kubelet/pods/222fb595-75f2-40c6-ba72-fcae616e3686/volumes" Nov 04 07:33:00.873540 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:33:00.873486 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:33:16.053141 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:16.053107 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:33:17.423895 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:17.423858 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:33:17.521336 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:17.521296 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:33:22.202357 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.202328 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:33:22.226047 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.226018 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-7ppvl"] Nov 04 07:33:22.226362 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.226350 5385 cpu_manager.go:414] "containerMap: removing container" podUID="222fb595-75f2-40c6-ba72-fcae616e3686" containerName="maas-api" Nov 04 07:33:22.226404 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.226363 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="222fb595-75f2-40c6-ba72-fcae616e3686" containerName="maas-api" Nov 04 07:33:22.226438 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.226431 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="222fb595-75f2-40c6-ba72-fcae616e3686" containerName="maas-api" Nov 04 07:33:22.229337 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.229316 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7ppvl" Nov 04 07:33:22.230267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.230247 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-wzjpf"] Nov 04 07:33:22.233254 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.233241 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-wzjpf" Nov 04 07:33:22.241026 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.239283 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Nov 04 07:33:22.241026 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.240885 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-7ppvl"] Nov 04 07:33:22.245993 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.245966 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27d5958b-d699-4846-b674-3bf2ef6d25ca-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-7ppvl\" (UID: \"27d5958b-d699-4846-b674-3bf2ef6d25ca\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7ppvl" Nov 04 07:33:22.246113 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.246016 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4522cb36-2dcc-483d-bf1e-42ffd1029cee-plugins-config-volume\") pod \"payload-processing-7bb9489676-wzjpf\" (UID: \"4522cb36-2dcc-483d-bf1e-42ffd1029cee\") " pod="openshift-ingress/payload-processing-7bb9489676-wzjpf" Nov 04 07:33:22.246172 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.246106 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-wzjpf"] Nov 04 07:33:22.246172 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.246146 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjszw\" (UniqueName: \"kubernetes.io/projected/27d5958b-d699-4846-b674-3bf2ef6d25ca-kube-api-access-hjszw\") pod \"payload-pre-processing-7b98bb678c-7ppvl\" (UID: \"27d5958b-d699-4846-b674-3bf2ef6d25ca\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7ppvl" Nov 04 07:33:22.246266 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.246233 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgv7f\" (UniqueName: \"kubernetes.io/projected/4522cb36-2dcc-483d-bf1e-42ffd1029cee-kube-api-access-cgv7f\") pod \"payload-processing-7bb9489676-wzjpf\" (UID: \"4522cb36-2dcc-483d-bf1e-42ffd1029cee\") " pod="openshift-ingress/payload-processing-7bb9489676-wzjpf" Nov 04 07:33:22.306227 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.306196 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:33:22.347442 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.347412 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27d5958b-d699-4846-b674-3bf2ef6d25ca-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-7ppvl\" (UID: \"27d5958b-d699-4846-b674-3bf2ef6d25ca\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7ppvl" Nov 04 07:33:22.347643 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.347461 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4522cb36-2dcc-483d-bf1e-42ffd1029cee-plugins-config-volume\") pod \"payload-processing-7bb9489676-wzjpf\" (UID: \"4522cb36-2dcc-483d-bf1e-42ffd1029cee\") " pod="openshift-ingress/payload-processing-7bb9489676-wzjpf" Nov 04 07:33:22.347643 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.347499 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjszw\" (UniqueName: \"kubernetes.io/projected/27d5958b-d699-4846-b674-3bf2ef6d25ca-kube-api-access-hjszw\") pod \"payload-pre-processing-7b98bb678c-7ppvl\" (UID: \"27d5958b-d699-4846-b674-3bf2ef6d25ca\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7ppvl" Nov 04 07:33:22.347643 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.347549 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgv7f\" (UniqueName: \"kubernetes.io/projected/4522cb36-2dcc-483d-bf1e-42ffd1029cee-kube-api-access-cgv7f\") pod \"payload-processing-7bb9489676-wzjpf\" (UID: \"4522cb36-2dcc-483d-bf1e-42ffd1029cee\") " pod="openshift-ingress/payload-processing-7bb9489676-wzjpf" Nov 04 07:33:22.348256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.348233 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4522cb36-2dcc-483d-bf1e-42ffd1029cee-plugins-config-volume\") pod \"payload-processing-7bb9489676-wzjpf\" (UID: \"4522cb36-2dcc-483d-bf1e-42ffd1029cee\") " pod="openshift-ingress/payload-processing-7bb9489676-wzjpf" Nov 04 07:33:22.348355 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.348255 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/27d5958b-d699-4846-b674-3bf2ef6d25ca-plugins-config-volume\") pod \"payload-pre-processing-7b98bb678c-7ppvl\" (UID: \"27d5958b-d699-4846-b674-3bf2ef6d25ca\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7ppvl" Nov 04 07:33:22.359830 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.359801 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgv7f\" (UniqueName: \"kubernetes.io/projected/4522cb36-2dcc-483d-bf1e-42ffd1029cee-kube-api-access-cgv7f\") pod \"payload-processing-7bb9489676-wzjpf\" (UID: \"4522cb36-2dcc-483d-bf1e-42ffd1029cee\") " pod="openshift-ingress/payload-processing-7bb9489676-wzjpf" Nov 04 07:33:22.359958 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.359867 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjszw\" (UniqueName: \"kubernetes.io/projected/27d5958b-d699-4846-b674-3bf2ef6d25ca-kube-api-access-hjszw\") pod \"payload-pre-processing-7b98bb678c-7ppvl\" (UID: \"27d5958b-d699-4846-b674-3bf2ef6d25ca\") " pod="openshift-ingress/payload-pre-processing-7b98bb678c-7ppvl" Nov 04 07:33:22.553582 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.553475 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7ppvl" Nov 04 07:33:22.559744 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.559720 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7bb9489676-wzjpf" Nov 04 07:33:22.831810 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.831567 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7b98bb678c-7ppvl"] Nov 04 07:33:22.834784 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:33:22.834714 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d5958b_d699_4846_b674_3bf2ef6d25ca.slice/crio-2b1daea51e3b535b1fe4d590fbf4ddf6dbb00753832fabd934d109949bf54d60 WatchSource:0}: Error finding container 2b1daea51e3b535b1fe4d590fbf4ddf6dbb00753832fabd934d109949bf54d60: Status 404 returned error can't find the container with id 2b1daea51e3b535b1fe4d590fbf4ddf6dbb00753832fabd934d109949bf54d60 Nov 04 07:33:22.836834 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.836804 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7bb9489676-wzjpf"] Nov 04 07:33:22.840315 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:33:22.840284 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4522cb36_2dcc_483d_bf1e_42ffd1029cee.slice/crio-09f3b54a8945a4bf6196b1cd0ecda19d423279ed458743e59f88ba2b41a86023 WatchSource:0}: Error finding container 09f3b54a8945a4bf6196b1cd0ecda19d423279ed458743e59f88ba2b41a86023: Status 404 returned error can't find the container with id 09f3b54a8945a4bf6196b1cd0ecda19d423279ed458743e59f88ba2b41a86023 Nov 04 07:33:22.850934 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.850900 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-wzjpf" event={"ID":"4522cb36-2dcc-483d-bf1e-42ffd1029cee","Type":"ContainerStarted","Data":"09f3b54a8945a4bf6196b1cd0ecda19d423279ed458743e59f88ba2b41a86023"} Nov 04 07:33:22.852199 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:22.852173 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7ppvl" event={"ID":"27d5958b-d699-4846-b674-3bf2ef6d25ca","Type":"ContainerStarted","Data":"2b1daea51e3b535b1fe4d590fbf4ddf6dbb00753832fabd934d109949bf54d60"} Nov 04 07:33:25.870500 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:25.870462 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7ppvl" event={"ID":"27d5958b-d699-4846-b674-3bf2ef6d25ca","Type":"ContainerStarted","Data":"c2618a71b6e79353986ed06a558c13ca42c1ceb498535884efa6e01ac8747dbf"} Nov 04 07:33:25.870982 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:25.870628 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7ppvl" Nov 04 07:33:25.872056 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:25.872034 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7bb9489676-wzjpf" event={"ID":"4522cb36-2dcc-483d-bf1e-42ffd1029cee","Type":"ContainerStarted","Data":"02b74b1d8781c2b9a7776b58928ad9f1e67d2f51c8de7a2eee334d60fa34f90d"} Nov 04 07:33:25.872174 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:25.872082 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7bb9489676-wzjpf" Nov 04 07:33:25.890490 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:25.890361 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7ppvl" podStartSLOduration=1.517689371 podStartE2EDuration="3.890346916s" totalImagesPullingTime="2.372657545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:33:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:33:25.889793434 +0000 UTC m=+1041.223733019" watchObservedRunningTime="2026-11-04 07:33:25.890346916 +0000 UTC m=+1041.224286501" Nov 04 07:33:25.908802 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:25.908758 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7bb9489676-wzjpf" podStartSLOduration=1.44939236 podStartE2EDuration="3.908748095s" totalImagesPullingTime="2.459355735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:33:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:33:25.907691389 +0000 UTC m=+1041.241630973" watchObservedRunningTime="2026-11-04 07:33:25.908748095 +0000 UTC m=+1041.242687669" Nov 04 07:33:30.874602 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:33:30.874566 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:33:36.879652 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:36.879625 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7bb9489676-wzjpf" Nov 04 07:33:36.880131 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:36.879684 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7b98bb678c-7ppvl" Nov 04 07:33:46.593705 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:46.593668 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:33:47.212017 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:47.211984 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:33:48.384632 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:48.384585 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:33:48.505772 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:48.505739 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:33:51.931098 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:51.931063 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc"] Nov 04 07:33:51.934330 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:51.934309 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:51.940006 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:51.939966 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Nov 04 07:33:51.940647 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:51.940630 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Nov 04 07:33:51.940756 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:51.940734 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-x7226\"" Nov 04 07:33:51.940837 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:51.940823 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Nov 04 07:33:51.946598 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:51.946575 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc"] Nov 04 07:33:52.061405 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.061364 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:33:52.099338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.099307 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46148a51-0296-47b0-abce-48b086c3d94e-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.099547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.099346 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46148a51-0296-47b0-abce-48b086c3d94e-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.099547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.099379 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46148a51-0296-47b0-abce-48b086c3d94e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.099681 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.099541 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46148a51-0296-47b0-abce-48b086c3d94e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.099681 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.099605 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ft9r\" (UniqueName: \"kubernetes.io/projected/46148a51-0296-47b0-abce-48b086c3d94e-kube-api-access-2ft9r\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.099681 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.099665 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46148a51-0296-47b0-abce-48b086c3d94e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.200417 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.200340 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ft9r\" (UniqueName: \"kubernetes.io/projected/46148a51-0296-47b0-abce-48b086c3d94e-kube-api-access-2ft9r\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.200417 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.200397 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46148a51-0296-47b0-abce-48b086c3d94e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.200667 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.200439 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46148a51-0296-47b0-abce-48b086c3d94e-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.200667 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.200458 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46148a51-0296-47b0-abce-48b086c3d94e-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.200667 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.200488 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46148a51-0296-47b0-abce-48b086c3d94e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.200667 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.200549 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46148a51-0296-47b0-abce-48b086c3d94e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.200888 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.200861 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/46148a51-0296-47b0-abce-48b086c3d94e-home\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.200983 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.200962 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/46148a51-0296-47b0-abce-48b086c3d94e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.201047 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.200977 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/46148a51-0296-47b0-abce-48b086c3d94e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.202843 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.202822 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/46148a51-0296-47b0-abce-48b086c3d94e-dshm\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.203121 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.203101 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/46148a51-0296-47b0-abce-48b086c3d94e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.216048 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.216025 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ft9r\" (UniqueName: \"kubernetes.io/projected/46148a51-0296-47b0-abce-48b086c3d94e-kube-api-access-2ft9r\") pod \"e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc\" (UID: \"46148a51-0296-47b0-abce-48b086c3d94e\") " pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.247971 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.247945 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:52.534820 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.534788 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc"] Nov 04 07:33:52.987961 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:52.987880 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" event={"ID":"46148a51-0296-47b0-abce-48b086c3d94e","Type":"ContainerStarted","Data":"25a92c001935a6d7cacee0fec8e44aab2eae5dc54b798ca474bd07f440025936"} Nov 04 07:33:53.903500 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:53.903472 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:33:53.993997 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:53.993956 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" event={"ID":"46148a51-0296-47b0-abce-48b086c3d94e","Type":"ContainerStarted","Data":"08142a798830975763162db2c6bd1852f9df1014b1a3a310f9aa658fd7bd5080"} Nov 04 07:33:54.015756 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:54.015706 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" podStartSLOduration=1.70988957 podStartE2EDuration="3.015687619s" totalImagesPullingTime="1.305798049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:33:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:33:54.014518194 +0000 UTC m=+1069.348457779" watchObservedRunningTime="2026-11-04 07:33:54.015687619 +0000 UTC m=+1069.349627203" Nov 04 07:33:54.249016 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:54.248979 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:33:56.632703 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.632666 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf"] Nov 04 07:33:56.636392 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.636369 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.641297 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.641272 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\"" Nov 04 07:33:56.647629 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.647607 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf"] Nov 04 07:33:56.687817 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.687786 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:33:56.740079 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.740050 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/526529db-d790-40e3-9983-bc2f8473a034-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.740225 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.740098 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rmsx\" (UniqueName: \"kubernetes.io/projected/526529db-d790-40e3-9983-bc2f8473a034-kube-api-access-6rmsx\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.740225 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.740119 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/526529db-d790-40e3-9983-bc2f8473a034-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.740225 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.740184 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/526529db-d790-40e3-9983-bc2f8473a034-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.740369 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.740248 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/526529db-d790-40e3-9983-bc2f8473a034-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.740369 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.740307 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/526529db-d790-40e3-9983-bc2f8473a034-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.841539 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.841490 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/526529db-d790-40e3-9983-bc2f8473a034-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.841711 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.841570 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rmsx\" (UniqueName: \"kubernetes.io/projected/526529db-d790-40e3-9983-bc2f8473a034-kube-api-access-6rmsx\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.841711 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.841591 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/526529db-d790-40e3-9983-bc2f8473a034-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.841711 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.841611 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/526529db-d790-40e3-9983-bc2f8473a034-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.841711 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.841632 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/526529db-d790-40e3-9983-bc2f8473a034-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.841711 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.841664 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/526529db-d790-40e3-9983-bc2f8473a034-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.841973 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.841940 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/526529db-d790-40e3-9983-bc2f8473a034-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.842049 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.842022 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/526529db-d790-40e3-9983-bc2f8473a034-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.842088 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.842070 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/526529db-d790-40e3-9983-bc2f8473a034-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.843827 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.843805 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/526529db-d790-40e3-9983-bc2f8473a034-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.844212 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.844194 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/526529db-d790-40e3-9983-bc2f8473a034-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.854165 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.854139 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rmsx\" (UniqueName: \"kubernetes.io/projected/526529db-d790-40e3-9983-bc2f8473a034-kube-api-access-6rmsx\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-4rscf\" (UID: \"526529db-d790-40e3-9983-bc2f8473a034\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:56.950574 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:56.950489 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:57.144544 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:57.144491 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:33:57.226479 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:57.226377 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf"] Nov 04 07:33:57.232439 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:33:57.232411 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod526529db_d790_40e3_9983_bc2f8473a034.slice/crio-77e67fbdff8973401f1cd1de2d6b7d69cacce814bc1fbdd4a3635303e73e903b WatchSource:0}: Error finding container 77e67fbdff8973401f1cd1de2d6b7d69cacce814bc1fbdd4a3635303e73e903b: Status 404 returned error can't find the container with id 77e67fbdff8973401f1cd1de2d6b7d69cacce814bc1fbdd4a3635303e73e903b Nov 04 07:33:58.014616 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:58.014582 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" event={"ID":"526529db-d790-40e3-9983-bc2f8473a034","Type":"ContainerStarted","Data":"72cf107be1e20c3f051049e53194a37a2bd29d5b6ead36ac42a412aba390e6e2"} Nov 04 07:33:58.015014 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:58.014631 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" event={"ID":"526529db-d790-40e3-9983-bc2f8473a034","Type":"ContainerStarted","Data":"77e67fbdff8973401f1cd1de2d6b7d69cacce814bc1fbdd4a3635303e73e903b"} Nov 04 07:33:58.049134 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:58.049096 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:33:58.951169 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:58.951132 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:33:59.287284 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.287186 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" podStartSLOduration=2.996322827 podStartE2EDuration="3.287170146s" totalImagesPullingTime="290.847319ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:33:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:33:58.068538212 +0000 UTC m=+1073.402477792" watchObservedRunningTime="2026-11-04 07:33:59.287170146 +0000 UTC m=+1074.621109731" Nov 04 07:33:59.287721 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.287381 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-q9dnj"] Nov 04 07:33:59.291273 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.291253 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.296875 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.296851 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\"" Nov 04 07:33:59.309163 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.309138 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-q9dnj"] Nov 04 07:33:59.344626 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.344602 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:33:59.367233 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.367203 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34c2083c-6a37-460e-b6bd-8510768d9e4e-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.367358 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.367261 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34c2083c-6a37-460e-b6bd-8510768d9e4e-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.367410 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.367353 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34c2083c-6a37-460e-b6bd-8510768d9e4e-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.367448 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.367417 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljdrc\" (UniqueName: \"kubernetes.io/projected/34c2083c-6a37-460e-b6bd-8510768d9e4e-kube-api-access-ljdrc\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.367482 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.367446 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34c2083c-6a37-460e-b6bd-8510768d9e4e-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.367580 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.367555 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34c2083c-6a37-460e-b6bd-8510768d9e4e-home\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.468199 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.468157 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljdrc\" (UniqueName: \"kubernetes.io/projected/34c2083c-6a37-460e-b6bd-8510768d9e4e-kube-api-access-ljdrc\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.468199 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.468205 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34c2083c-6a37-460e-b6bd-8510768d9e4e-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.468431 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.468221 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34c2083c-6a37-460e-b6bd-8510768d9e4e-home\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.468431 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.468261 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34c2083c-6a37-460e-b6bd-8510768d9e4e-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.468431 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.468303 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34c2083c-6a37-460e-b6bd-8510768d9e4e-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.468431 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.468345 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34c2083c-6a37-460e-b6bd-8510768d9e4e-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.468769 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.468745 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/34c2083c-6a37-460e-b6bd-8510768d9e4e-home\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.468862 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.468794 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/34c2083c-6a37-460e-b6bd-8510768d9e4e-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.468862 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.468837 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/34c2083c-6a37-460e-b6bd-8510768d9e4e-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.470513 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.470484 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/34c2083c-6a37-460e-b6bd-8510768d9e4e-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.470769 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.470749 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/34c2083c-6a37-460e-b6bd-8510768d9e4e-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.482303 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.482275 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljdrc\" (UniqueName: \"kubernetes.io/projected/34c2083c-6a37-460e-b6bd-8510768d9e4e-kube-api-access-ljdrc\") pod \"echo-simulator-kserve-64b6f8f96c-q9dnj\" (UID: \"34c2083c-6a37-460e-b6bd-8510768d9e4e\") " pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.607639 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.607560 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:33:59.884777 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:33:59.884742 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-q9dnj"] Nov 04 07:33:59.887724 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:33:59.887698 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34c2083c_6a37_460e_b6bd_8510768d9e4e.slice/crio-fba807253f4e027fd45a62c3268071f22e07e0d1461250896ab34fee6fe13062 WatchSource:0}: Error finding container fba807253f4e027fd45a62c3268071f22e07e0d1461250896ab34fee6fe13062: Status 404 returned error can't find the container with id fba807253f4e027fd45a62c3268071f22e07e0d1461250896ab34fee6fe13062 Nov 04 07:34:00.026428 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.026380 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" event={"ID":"34c2083c-6a37-460e-b6bd-8510768d9e4e","Type":"ContainerStarted","Data":"fba807253f4e027fd45a62c3268071f22e07e0d1461250896ab34fee6fe13062"} Nov 04 07:34:00.522256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.522205 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2"] Nov 04 07:34:00.525959 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.525934 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.547627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.547559 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2"] Nov 04 07:34:00.550038 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.549468 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Nov 04 07:34:00.593372 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.593334 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:00.679425 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.679376 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.679779 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.679443 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.679779 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.679557 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.679779 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.679584 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.679779 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.679618 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.680017 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.679778 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvhsf\" (UniqueName: \"kubernetes.io/projected/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-kube-api-access-hvhsf\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.781468 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.781369 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.781468 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.781434 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.781468 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.781469 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.781754 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.781560 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvhsf\" (UniqueName: \"kubernetes.io/projected/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-kube-api-access-hvhsf\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.781754 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.781623 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.781754 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.781682 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.781916 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.781747 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.782081 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.782058 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-home\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.782632 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.782607 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.784578 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.784553 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-dshm\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.786414 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.786388 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.795885 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.795848 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvhsf\" (UniqueName: \"kubernetes.io/projected/092adbcf-64c3-4ac6-80a7-9ce2d00e675a-kube-api-access-hvhsf\") pod \"facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2\" (UID: \"092adbcf-64c3-4ac6-80a7-9ce2d00e675a\") " pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.842804 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:00.842614 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:00.887750 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:34:00.887714 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:34:01.153319 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:01.153287 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2"] Nov 04 07:34:01.257826 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:01.257290 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:02.037226 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:02.037191 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" event={"ID":"34c2083c-6a37-460e-b6bd-8510768d9e4e","Type":"ContainerStarted","Data":"554168ba74bd633b8fed05178634cf2b2e7c3024e82f894e06c0912473ff0cc8"} Nov 04 07:34:02.038765 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:02.038729 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" event={"ID":"092adbcf-64c3-4ac6-80a7-9ce2d00e675a","Type":"ContainerStarted","Data":"7a93c6e76aec98266bc4e36d528125c8cfb243ccaff2809a70297cb933c30f59"} Nov 04 07:34:02.038905 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:02.038781 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" event={"ID":"092adbcf-64c3-4ac6-80a7-9ce2d00e675a","Type":"ContainerStarted","Data":"35fa4d57d619e58b7bd0562462d4bde9f9bbf09def8660efdffa75ba7ca90d31"} Nov 04 07:34:02.058019 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:02.057964 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" podStartSLOduration=1.013563355 podStartE2EDuration="3.057947286s" totalImagesPullingTime="2.044383931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:02.05767093 +0000 UTC m=+1077.391610526" watchObservedRunningTime="2026-11-04 07:34:02.057947286 +0000 UTC m=+1077.391886871" Nov 04 07:34:02.079856 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:02.079801 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" podStartSLOduration=1.750449972 podStartE2EDuration="2.07978281s" totalImagesPullingTime="329.332838ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:02.077673802 +0000 UTC m=+1077.411613387" watchObservedRunningTime="2026-11-04 07:34:02.07978281 +0000 UTC m=+1077.413722396" Nov 04 07:34:02.248770 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:02.248673 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:34:02.261669 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:02.261631 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:34:02.608583 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:02.608479 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:34:02.842966 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:02.842938 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:03.054750 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:03.054719 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc" Nov 04 07:34:06.951393 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:06.951355 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:34:06.963992 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:06.963962 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:34:07.071301 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:07.071270 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-4rscf" Nov 04 07:34:09.608397 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:09.608359 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:34:09.620754 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:09.620731 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:34:10.091348 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:10.091317 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-q9dnj" Nov 04 07:34:10.843152 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:10.843121 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:10.855447 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:10.855422 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:11.096560 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:11.096469 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2" Nov 04 07:34:15.682663 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.682625 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v"] Nov 04 07:34:15.686577 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.686554 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.692119 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.692097 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2rn5b\"" Nov 04 07:34:15.703793 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.703769 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v"] Nov 04 07:34:15.820478 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.820427 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae22c880-563f-4c2c-9461-b1e185adf358-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.820670 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.820541 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae22c880-563f-4c2c-9461-b1e185adf358-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.820670 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.820600 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae22c880-563f-4c2c-9461-b1e185adf358-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.820744 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.820728 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae22c880-563f-4c2c-9461-b1e185adf358-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.820791 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.820775 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae22c880-563f-4c2c-9461-b1e185adf358-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.820874 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.820858 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae22c880-563f-4c2c-9461-b1e185adf358-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.820947 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.820935 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae22c880-563f-4c2c-9461-b1e185adf358-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.820990 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.820975 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7zxw\" (UniqueName: \"kubernetes.io/projected/ae22c880-563f-4c2c-9461-b1e185adf358-kube-api-access-x7zxw\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.821023 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.820998 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae22c880-563f-4c2c-9461-b1e185adf358-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.922404 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.922317 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae22c880-563f-4c2c-9461-b1e185adf358-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.922404 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.922367 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae22c880-563f-4c2c-9461-b1e185adf358-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.922404 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.922390 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae22c880-563f-4c2c-9461-b1e185adf358-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.922729 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.922433 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae22c880-563f-4c2c-9461-b1e185adf358-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.922729 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.922476 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7zxw\" (UniqueName: \"kubernetes.io/projected/ae22c880-563f-4c2c-9461-b1e185adf358-kube-api-access-x7zxw\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.922729 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.922532 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae22c880-563f-4c2c-9461-b1e185adf358-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.922888 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.922781 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae22c880-563f-4c2c-9461-b1e185adf358-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.922888 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.922803 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae22c880-563f-4c2c-9461-b1e185adf358-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.922888 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.922826 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae22c880-563f-4c2c-9461-b1e185adf358-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.923035 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.922931 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae22c880-563f-4c2c-9461-b1e185adf358-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.923103 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.923084 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae22c880-563f-4c2c-9461-b1e185adf358-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.923154 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.923110 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae22c880-563f-4c2c-9461-b1e185adf358-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.923305 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.923282 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae22c880-563f-4c2c-9461-b1e185adf358-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.923346 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.923288 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae22c880-563f-4c2c-9461-b1e185adf358-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.925035 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.925010 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae22c880-563f-4c2c-9461-b1e185adf358-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.925197 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.925173 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae22c880-563f-4c2c-9461-b1e185adf358-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.940306 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.940246 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae22c880-563f-4c2c-9461-b1e185adf358-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.940405 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.940321 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7zxw\" (UniqueName: \"kubernetes.io/projected/ae22c880-563f-4c2c-9461-b1e185adf358-kube-api-access-x7zxw\") pod \"maas-default-gateway-openshift-default-74854d8ccf-lhm7v\" (UID: \"ae22c880-563f-4c2c-9461-b1e185adf358\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:15.999105 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:15.999064 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:16.115890 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:16.115855 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:16.209338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:16.209256 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5ffbd6df97-qw2pm"] Nov 04 07:34:16.209538 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:16.209492 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-5ffbd6df97-qw2pm" podUID="2fc5119a-070d-4016-8124-ba6a546d9e35" containerName="authorino" containerID="cri-o://fe3bdcc0cb7600310d01ff79c98f5dbc05e39279940441639df123f519dbf047" gracePeriod=30 Nov 04 07:34:16.287198 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:16.287173 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v"] Nov 04 07:34:16.306463 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:34:16.306433 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae22c880_563f_4c2c_9461_b1e185adf358.slice/crio-23cf3945a4d9d7acb005242771044037130c88722a2994cddec28de4dd44f4d7 WatchSource:0}: Error finding container 23cf3945a4d9d7acb005242771044037130c88722a2994cddec28de4dd44f4d7: Status 404 returned error can't find the container with id 23cf3945a4d9d7acb005242771044037130c88722a2994cddec28de4dd44f4d7 Nov 04 07:34:16.452453 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:16.452428 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ffbd6df97-qw2pm" Nov 04 07:34:16.629284 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:16.629253 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2fc5119a-070d-4016-8124-ba6a546d9e35-tls-cert\" (UniqueName: \"kubernetes.io/secret/2fc5119a-070d-4016-8124-ba6a546d9e35-tls-cert\") pod \"2fc5119a-070d-4016-8124-ba6a546d9e35\" (UID: \"2fc5119a-070d-4016-8124-ba6a546d9e35\") " Nov 04 07:34:16.629450 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:16.629293 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fc5119a-070d-4016-8124-ba6a546d9e35-kube-api-access-r5pf9\" (UniqueName: \"kubernetes.io/projected/2fc5119a-070d-4016-8124-ba6a546d9e35-kube-api-access-r5pf9\") pod \"2fc5119a-070d-4016-8124-ba6a546d9e35\" (UID: \"2fc5119a-070d-4016-8124-ba6a546d9e35\") " Nov 04 07:34:16.631290 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:16.631265 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fc5119a-070d-4016-8124-ba6a546d9e35-kube-api-access-r5pf9" pod "2fc5119a-070d-4016-8124-ba6a546d9e35" (UID: "2fc5119a-070d-4016-8124-ba6a546d9e35"). InnerVolumeSpecName "kube-api-access-r5pf9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:34:16.639275 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:16.639249 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fc5119a-070d-4016-8124-ba6a546d9e35-tls-cert" pod "2fc5119a-070d-4016-8124-ba6a546d9e35" (UID: "2fc5119a-070d-4016-8124-ba6a546d9e35"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:34:16.730566 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:16.730529 5385 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2fc5119a-070d-4016-8124-ba6a546d9e35-tls-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:34:16.730566 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:16.730562 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5pf9\" (UniqueName: \"kubernetes.io/projected/2fc5119a-070d-4016-8124-ba6a546d9e35-kube-api-access-r5pf9\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:34:17.112672 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:17.112634 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" event={"ID":"ae22c880-563f-4c2c-9461-b1e185adf358","Type":"ContainerStarted","Data":"23cf3945a4d9d7acb005242771044037130c88722a2994cddec28de4dd44f4d7"} Nov 04 07:34:17.113836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:17.113799 5385 generic.go:356] "Generic (PLEG): container finished" podID="2fc5119a-070d-4016-8124-ba6a546d9e35" containerID="fe3bdcc0cb7600310d01ff79c98f5dbc05e39279940441639df123f519dbf047" exitCode=0 Nov 04 07:34:17.113957 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:17.113843 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5ffbd6df97-qw2pm" Nov 04 07:34:17.113957 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:17.113900 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ffbd6df97-qw2pm" event={"ID":"2fc5119a-070d-4016-8124-ba6a546d9e35","Type":"ContainerDied","Data":"fe3bdcc0cb7600310d01ff79c98f5dbc05e39279940441639df123f519dbf047"} Nov 04 07:34:17.113957 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:17.113933 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5ffbd6df97-qw2pm" event={"ID":"2fc5119a-070d-4016-8124-ba6a546d9e35","Type":"ContainerDied","Data":"b8502d2775270ae49a44c6125b496d990ada46446137f4236bdf3ae7a60a5066"} Nov 04 07:34:17.113957 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:17.113953 5385 scope.go:122] "RemoveContainer" containerID="fe3bdcc0cb7600310d01ff79c98f5dbc05e39279940441639df123f519dbf047" Nov 04 07:34:17.122381 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:17.122359 5385 scope.go:122] "RemoveContainer" containerID="fe3bdcc0cb7600310d01ff79c98f5dbc05e39279940441639df123f519dbf047" Nov 04 07:34:17.122660 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:34:17.122640 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe3bdcc0cb7600310d01ff79c98f5dbc05e39279940441639df123f519dbf047\": container with ID starting with fe3bdcc0cb7600310d01ff79c98f5dbc05e39279940441639df123f519dbf047 not found: ID does not exist" containerID="fe3bdcc0cb7600310d01ff79c98f5dbc05e39279940441639df123f519dbf047" Nov 04 07:34:17.122721 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:17.122668 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe3bdcc0cb7600310d01ff79c98f5dbc05e39279940441639df123f519dbf047"} err="failed to get container status \"fe3bdcc0cb7600310d01ff79c98f5dbc05e39279940441639df123f519dbf047\": rpc error: code = NotFound desc = could not find container \"fe3bdcc0cb7600310d01ff79c98f5dbc05e39279940441639df123f519dbf047\": container with ID starting with fe3bdcc0cb7600310d01ff79c98f5dbc05e39279940441639df123f519dbf047 not found: ID does not exist" Nov 04 07:34:17.201444 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:17.201354 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5ffbd6df97-qw2pm"] Nov 04 07:34:17.312571 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:17.312539 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5ffbd6df97-qw2pm"] Nov 04 07:34:17.379448 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:17.379415 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:17.563929 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:17.563839 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:18.812739 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:18.812694 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:18.813159 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:18.812931 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2fc5119a-070d-4016-8124-ba6a546d9e35" path="/var/lib/kubelet/pods/2fc5119a-070d-4016-8124-ba6a546d9e35/volumes" Nov 04 07:34:18.942876 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:18.942844 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:19.395232 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:19.395197 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:19.395343 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:19.395267 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:19.395343 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:19.395301 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:20.134826 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:20.134791 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" event={"ID":"ae22c880-563f-4c2c-9461-b1e185adf358","Type":"ContainerStarted","Data":"80deddce641354a87f104498071286f1cd042041a893bf2270c8096faf79c5b3"} Nov 04 07:34:20.168753 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:20.168680 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" podStartSLOduration=2.089851575 podStartE2EDuration="5.168648302s" totalImagesPullingTime="3.078796727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:20.166728215 +0000 UTC m=+1095.500667802" watchObservedRunningTime="2026-11-04 07:34:20.168648302 +0000 UTC m=+1095.502587882" Nov 04 07:34:21.000047 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:21.000015 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:21.001557 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:21.001518 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-lhm7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 07:34:21.001718 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:21.001579 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" podUID="ae22c880-563f-4c2c-9461-b1e185adf358" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 07:34:21.999950 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:21.999912 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-lhm7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 07:34:22.000342 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:21.999971 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" podUID="ae22c880-563f-4c2c-9461-b1e185adf358" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 07:34:22.999444 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:22.999409 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-lhm7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 07:34:22.999650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:22.999469 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" podUID="ae22c880-563f-4c2c-9461-b1e185adf358" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 07:34:23.999866 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:23.999830 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-lhm7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 07:34:24.000358 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:23.999889 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" podUID="ae22c880-563f-4c2c-9461-b1e185adf358" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 07:34:25.000250 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:25.000207 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-lhm7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 07:34:25.000691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:25.000265 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" podUID="ae22c880-563f-4c2c-9461-b1e185adf358" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 07:34:25.999943 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:25.999906 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-lhm7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 07:34:26.000124 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:25.999962 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" podUID="ae22c880-563f-4c2c-9461-b1e185adf358" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 07:34:26.999381 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:26.999344 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-lhm7v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Nov 04 07:34:26.999795 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:26.999396 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" podUID="ae22c880-563f-4c2c-9461-b1e185adf358" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Nov 04 07:34:28.004093 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:28.004064 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:28.004566 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:28.004336 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:28.004920 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:28.004901 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" Nov 04 07:34:30.711155 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.711117 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn"] Nov 04 07:34:30.711634 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.711466 5385 cpu_manager.go:414] "containerMap: removing container" podUID="2fc5119a-070d-4016-8124-ba6a546d9e35" containerName="authorino" Nov 04 07:34:30.711634 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.711476 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2fc5119a-070d-4016-8124-ba6a546d9e35" containerName="authorino" Nov 04 07:34:30.711634 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.711570 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="2fc5119a-070d-4016-8124-ba6a546d9e35" containerName="authorino" Nov 04 07:34:30.717918 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.717896 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.732085 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.732062 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn"] Nov 04 07:34:30.758212 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.758186 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3c860e0-8952-4f74-acd1-803a8a6acc85-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.758347 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.758219 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3c860e0-8952-4f74-acd1-803a8a6acc85-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.758347 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.758243 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3c860e0-8952-4f74-acd1-803a8a6acc85-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.758347 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.758258 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3c860e0-8952-4f74-acd1-803a8a6acc85-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.758449 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.758359 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3c860e0-8952-4f74-acd1-803a8a6acc85-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.758483 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.758458 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3c860e0-8952-4f74-acd1-803a8a6acc85-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.758541 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.758525 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3c860e0-8952-4f74-acd1-803a8a6acc85-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.758574 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.758559 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3c860e0-8952-4f74-acd1-803a8a6acc85-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.758610 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.758584 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqxxh\" (UniqueName: \"kubernetes.io/projected/c3c860e0-8952-4f74-acd1-803a8a6acc85-kube-api-access-vqxxh\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.859167 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.859141 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3c860e0-8952-4f74-acd1-803a8a6acc85-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.859352 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.859171 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3c860e0-8952-4f74-acd1-803a8a6acc85-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.859352 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.859194 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqxxh\" (UniqueName: \"kubernetes.io/projected/c3c860e0-8952-4f74-acd1-803a8a6acc85-kube-api-access-vqxxh\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.859352 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.859234 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3c860e0-8952-4f74-acd1-803a8a6acc85-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.859352 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.859250 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3c860e0-8952-4f74-acd1-803a8a6acc85-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.859352 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.859272 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3c860e0-8952-4f74-acd1-803a8a6acc85-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.859352 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.859295 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3c860e0-8952-4f74-acd1-803a8a6acc85-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.859352 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.859327 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3c860e0-8952-4f74-acd1-803a8a6acc85-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.859777 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.859743 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3c860e0-8952-4f74-acd1-803a8a6acc85-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.859863 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.859845 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3c860e0-8952-4f74-acd1-803a8a6acc85-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.859970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.859942 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c3c860e0-8952-4f74-acd1-803a8a6acc85-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.860095 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.860066 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c3c860e0-8952-4f74-acd1-803a8a6acc85-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.860157 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.860133 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c3c860e0-8952-4f74-acd1-803a8a6acc85-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.860211 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.860154 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c3c860e0-8952-4f74-acd1-803a8a6acc85-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.861483 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.861462 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c3c860e0-8952-4f74-acd1-803a8a6acc85-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.861936 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.861912 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c3c860e0-8952-4f74-acd1-803a8a6acc85-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.874016 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.873992 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c3c860e0-8952-4f74-acd1-803a8a6acc85-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.875103 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:30.875077 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqxxh\" (UniqueName: \"kubernetes.io/projected/c3c860e0-8952-4f74-acd1-803a8a6acc85-kube-api-access-vqxxh\") pod \"maas-default-gateway-openshift-default-74854d8ccf-7wfxn\" (UID: \"c3c860e0-8952-4f74-acd1-803a8a6acc85\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:30.876482 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:34:30.876456 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:34:31.030916 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:31.030834 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:31.317718 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:31.317637 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn"] Nov 04 07:34:31.321060 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:34:31.321032 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3c860e0_8952_4f74_acd1_803a8a6acc85.slice/crio-83662bda116511b74895b2d475cc7b797cb548ec9604b913cc87a8a3b0b0eeb5 WatchSource:0}: Error finding container 83662bda116511b74895b2d475cc7b797cb548ec9604b913cc87a8a3b0b0eeb5: Status 404 returned error can't find the container with id 83662bda116511b74895b2d475cc7b797cb548ec9604b913cc87a8a3b0b0eeb5 Nov 04 07:34:31.323948 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:31.323912 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:31.324070 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:31.323977 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:31.324070 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:31.324010 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:32.209048 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:32.209011 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" event={"ID":"c3c860e0-8952-4f74-acd1-803a8a6acc85","Type":"ContainerStarted","Data":"d4cf82ff3971df4e7560f2b2b63e22adce69509eaaf996995950722f18f5c104"} Nov 04 07:34:32.209569 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:32.209062 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" event={"ID":"c3c860e0-8952-4f74-acd1-803a8a6acc85","Type":"ContainerStarted","Data":"83662bda116511b74895b2d475cc7b797cb548ec9604b913cc87a8a3b0b0eeb5"} Nov 04 07:34:33.032036 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:33.031995 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:33.033516 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:33.033474 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-7wfxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 04 07:34:33.033710 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:33.033554 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" podUID="c3c860e0-8952-4f74-acd1-803a8a6acc85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 04 07:34:34.031704 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:34.031668 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-7wfxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 04 07:34:34.032090 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:34.031740 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" podUID="c3c860e0-8952-4f74-acd1-803a8a6acc85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 04 07:34:35.031081 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:35.031041 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-7wfxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 04 07:34:35.031296 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:35.031108 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" podUID="c3c860e0-8952-4f74-acd1-803a8a6acc85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 04 07:34:36.031680 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:36.031638 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-7wfxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 04 07:34:36.032166 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:36.031701 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" podUID="c3c860e0-8952-4f74-acd1-803a8a6acc85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 04 07:34:37.032174 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:37.032136 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-7wfxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 04 07:34:37.032602 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:37.032196 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" podUID="c3c860e0-8952-4f74-acd1-803a8a6acc85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 04 07:34:38.031320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:38.031271 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-7wfxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Nov 04 07:34:38.031544 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:38.031339 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" podUID="c3c860e0-8952-4f74-acd1-803a8a6acc85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Nov 04 07:34:39.760061 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:39.760025 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:39.760548 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:39.760459 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:39.761213 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:39.761187 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" Nov 04 07:34:39.786363 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:39.786314 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-7wfxn" podStartSLOduration=9.786298839 podStartE2EDuration="9.786298839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:32.234529593 +0000 UTC m=+1107.568469194" watchObservedRunningTime="2026-11-04 07:34:39.786298839 +0000 UTC m=+1115.120238422" Nov 04 07:34:43.429630 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.429601 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:43.555081 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.555040 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:43.718548 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.718460 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:43.832696 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.832662 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9"] Nov 04 07:34:43.839425 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.839393 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.849913 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.849889 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-60ed4a-openshift-default-dockercfg-dcrsd\"" Nov 04 07:34:43.851562 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.851533 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9"] Nov 04 07:34:43.886881 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.886847 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.887040 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.886886 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.887040 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.886954 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.887040 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.886975 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xwt2\" (UniqueName: \"kubernetes.io/projected/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-kube-api-access-8xwt2\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.887040 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.887016 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.887230 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.887120 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.887230 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.887219 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.887323 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.887285 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.887361 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.887319 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.945195 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.945164 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:43.988132 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.988045 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.988132 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.988091 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.988340 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.988181 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.988340 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.988209 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.988340 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.988237 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.988340 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.988261 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xwt2\" (UniqueName: \"kubernetes.io/projected/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-kube-api-access-8xwt2\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.988340 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.988308 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.988340 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.988339 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.988675 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.988380 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.988675 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.988537 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.988675 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.988603 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.988844 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.988747 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.988939 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.988916 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.989218 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.989196 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.990875 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.990847 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:43.990989 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:43.990925 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:44.004171 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.004143 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw"] Nov 04 07:34:44.008096 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.008076 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.017497 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.017460 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-016e6c-openshift-default-dockercfg-d2ms9\"" Nov 04 07:34:44.018348 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.018323 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:44.024480 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.024454 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xwt2\" (UniqueName: \"kubernetes.io/projected/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-kube-api-access-8xwt2\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:44.024721 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.024696 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw"] Nov 04 07:34:44.081681 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.081653 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:44.089041 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.089013 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.089170 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.089059 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.089228 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.089179 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.089284 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.089226 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq46w\" (UniqueName: \"kubernetes.io/projected/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-kube-api-access-lq46w\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.089284 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.089274 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.089387 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.089301 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.089387 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.089341 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.089387 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.089365 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.089543 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.089390 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.159220 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.159188 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:44.161249 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.161219 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4"] Nov 04 07:34:44.164640 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.164621 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.182530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.182425 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-1f6ce4-openshift-default-dockercfg-vncb8\"" Nov 04 07:34:44.184046 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.184019 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4"] Nov 04 07:34:44.190073 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.190051 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.190212 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.190084 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.190212 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.190118 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.190212 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.190157 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.190212 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.190175 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.190212 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.190204 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.190520 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.190249 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq46w\" (UniqueName: \"kubernetes.io/projected/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-kube-api-access-lq46w\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.190520 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.190312 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.190520 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.190352 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.190689 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.190557 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.190798 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.190772 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.191106 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.191082 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.191419 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.191391 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.191694 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.191674 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.193364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.193338 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.193737 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.193716 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.196067 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.196045 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:44.206717 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.206692 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.206949 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.206926 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq46w\" (UniqueName: \"kubernetes.io/projected/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-kube-api-access-lq46w\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.291640 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.291472 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-token\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.291822 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.291634 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-workload-socket\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.291822 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.291672 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-envoy\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.291947 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.291828 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkq74\" (UniqueName: \"kubernetes.io/projected/d990bf05-77a4-4ebd-a301-c55f5bb7d526-kube-api-access-bkq74\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.291947 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.291880 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-credential-socket\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.291947 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.291917 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-data\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.292099 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.291957 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-workload-certs\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.292099 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.292000 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-podinfo\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.292099 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.292043 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istiod-ca-cert\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.322194 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.322159 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:44.330874 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.330841 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:44.393061 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.393029 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-data\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.393219 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.393084 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-workload-certs\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.393219 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.393126 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-podinfo\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.393219 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.393165 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istiod-ca-cert\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.393219 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.393212 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-token\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.393435 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.393247 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-workload-socket\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.393435 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.393275 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-envoy\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.393435 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.393382 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkq74\" (UniqueName: \"kubernetes.io/projected/d990bf05-77a4-4ebd-a301-c55f5bb7d526-kube-api-access-bkq74\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.393435 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.393419 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-credential-socket\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.393679 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.393584 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-data\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.393679 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.393609 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-workload-certs\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.393792 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.393776 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-credential-socket\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.393853 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.393823 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-workload-socket\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.394365 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.394318 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istiod-ca-cert\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.396084 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.396064 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-podinfo\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.396335 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.396315 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-envoy\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.406333 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.406311 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-token\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.406449 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.406392 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkq74\" (UniqueName: \"kubernetes.io/projected/d990bf05-77a4-4ebd-a301-c55f5bb7d526-kube-api-access-bkq74\") pod \"e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.435906 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.435881 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:44.464736 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.464703 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9"] Nov 04 07:34:44.469114 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:34:44.469085 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b7e3053_63a4_43c2_ada7_f98132ef6d4d.slice/crio-c0c1f35b4c932eec263f7d30ef2683d5c421f8aa830af133008de4d53f5422d1 WatchSource:0}: Error finding container c0c1f35b4c932eec263f7d30ef2683d5c421f8aa830af133008de4d53f5422d1: Status 404 returned error can't find the container with id c0c1f35b4c932eec263f7d30ef2683d5c421f8aa830af133008de4d53f5422d1 Nov 04 07:34:44.471788 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.471751 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:44.471862 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.471839 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:44.471904 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.471886 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:44.490290 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.490265 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:44.613649 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.613484 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw"] Nov 04 07:34:44.615545 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:34:44.615517 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a2a80ac_2bde_4025_9a4f_db2daa2ba28a.slice/crio-fb15afeabd4718f0c4e4e4fc429448ef0fabbcb1d831f0f404fc54d4406d0559 WatchSource:0}: Error finding container fb15afeabd4718f0c4e4e4fc429448ef0fabbcb1d831f0f404fc54d4406d0559: Status 404 returned error can't find the container with id fb15afeabd4718f0c4e4e4fc429448ef0fabbcb1d831f0f404fc54d4406d0559 Nov 04 07:34:44.618252 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.618222 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:44.618351 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.618294 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:44.618351 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.618338 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:44.782283 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.782256 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4"] Nov 04 07:34:44.785190 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:34:44.785160 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd990bf05_77a4_4ebd_a301_c55f5bb7d526.slice/crio-21ca19cdafc2002cb1f520532f15c1dce06eec948200cf78956ee30ee01f2cf5 WatchSource:0}: Error finding container 21ca19cdafc2002cb1f520532f15c1dce06eec948200cf78956ee30ee01f2cf5: Status 404 returned error can't find the container with id 21ca19cdafc2002cb1f520532f15c1dce06eec948200cf78956ee30ee01f2cf5 Nov 04 07:34:44.788015 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.787978 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:44.788101 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.788058 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:44.788147 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:44.788111 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:45.272327 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:45.272291 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" event={"ID":"d990bf05-77a4-4ebd-a301-c55f5bb7d526","Type":"ContainerStarted","Data":"cb66ca294a82a3c4a5dd785ba1077f0b517b1f192067e0556b1896fe8269b2e7"} Nov 04 07:34:45.272524 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:45.272386 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" event={"ID":"d990bf05-77a4-4ebd-a301-c55f5bb7d526","Type":"ContainerStarted","Data":"21ca19cdafc2002cb1f520532f15c1dce06eec948200cf78956ee30ee01f2cf5"} Nov 04 07:34:45.273886 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:45.273848 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" event={"ID":"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a","Type":"ContainerStarted","Data":"b1688dc4967fb416014424c6cf890f716885edfa81f525a40b96a3c97cf9cfc8"} Nov 04 07:34:45.273886 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:45.273892 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" event={"ID":"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a","Type":"ContainerStarted","Data":"fb15afeabd4718f0c4e4e4fc429448ef0fabbcb1d831f0f404fc54d4406d0559"} Nov 04 07:34:45.275277 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:45.275255 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" event={"ID":"6b7e3053-63a4-43c2-ada7-f98132ef6d4d","Type":"ContainerStarted","Data":"05a388de22fde529f354962c82f3dfe918023fd743d50c458d4f22b225143b5f"} Nov 04 07:34:45.275406 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:45.275291 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" event={"ID":"6b7e3053-63a4-43c2-ada7-f98132ef6d4d","Type":"ContainerStarted","Data":"c0c1f35b4c932eec263f7d30ef2683d5c421f8aa830af133008de4d53f5422d1"} Nov 04 07:34:45.296985 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:45.296945 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" podStartSLOduration=1.296933187 podStartE2EDuration="1.296933187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:45.295917336 +0000 UTC m=+1120.629856921" watchObservedRunningTime="2026-11-04 07:34:45.296933187 +0000 UTC m=+1120.630872770" Nov 04 07:34:45.319625 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:45.319583 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" podStartSLOduration=2.319570664 podStartE2EDuration="2.319570664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:45.318628714 +0000 UTC m=+1120.652568345" watchObservedRunningTime="2026-11-04 07:34:45.319570664 +0000 UTC m=+1120.653510226" Nov 04 07:34:45.322617 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:45.322601 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:45.327762 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:45.327742 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:45.344052 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:45.344009 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" podStartSLOduration=2.343997811 podStartE2EDuration="2.343997811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:45.342567069 +0000 UTC m=+1120.676506650" watchObservedRunningTime="2026-11-04 07:34:45.343997811 +0000 UTC m=+1120.677937460" Nov 04 07:34:45.491003 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:45.490966 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:45.495936 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:45.495913 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:45.731405 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:45.731374 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:46.128436 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.128390 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq"] Nov 04 07:34:46.132154 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.132133 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.143898 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.143878 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-df3be8-openshift-default-dockercfg-xkvb9\"" Nov 04 07:34:46.148088 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.148064 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq"] Nov 04 07:34:46.159949 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.159928 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:46.165230 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.165206 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:46.181080 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.181054 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:46.217579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.216433 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.217579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.216581 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.217579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.216613 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf5gg\" (UniqueName: \"kubernetes.io/projected/2b293d12-35ef-47df-9c77-27f734441df4-kube-api-access-pf5gg\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.217579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.216647 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.217579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.216673 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b293d12-35ef-47df-9c77-27f734441df4-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.217579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.216747 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b293d12-35ef-47df-9c77-27f734441df4-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.217579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.216838 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.217579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.216866 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b293d12-35ef-47df-9c77-27f734441df4-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.217579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.216965 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.279943 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.279913 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:46.280142 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.279965 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:46.280142 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.279991 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:46.280635 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.280612 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:34:46.280888 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.280873 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:34:46.280937 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.280918 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:34:46.320535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.320054 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.320535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.320105 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b293d12-35ef-47df-9c77-27f734441df4-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.320535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.320169 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.320535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.320209 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.320535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.320237 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.320535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.320261 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf5gg\" (UniqueName: \"kubernetes.io/projected/2b293d12-35ef-47df-9c77-27f734441df4-kube-api-access-pf5gg\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.320535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.320294 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.320535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.320322 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b293d12-35ef-47df-9c77-27f734441df4-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.320535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.320394 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b293d12-35ef-47df-9c77-27f734441df4-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.321939 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.321169 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b293d12-35ef-47df-9c77-27f734441df4-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.321939 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.321484 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.322169 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.322138 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.322169 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.322160 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.322717 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.322689 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.330207 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.328181 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.330207 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.328932 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b293d12-35ef-47df-9c77-27f734441df4-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.336471 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.336442 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf5gg\" (UniqueName: \"kubernetes.io/projected/2b293d12-35ef-47df-9c77-27f734441df4-kube-api-access-pf5gg\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.338829 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.337909 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b293d12-35ef-47df-9c77-27f734441df4-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.352301 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.352270 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:46.446038 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.445952 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:46.732126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.732102 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq"] Nov 04 07:34:46.734760 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:34:46.734735 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b293d12_35ef_47df_9c77_27f734441df4.slice/crio-b188858963392e946285fe7b0b1160e14cf23e96f0e8d80cff362a5b4819e93a WatchSource:0}: Error finding container b188858963392e946285fe7b0b1160e14cf23e96f0e8d80cff362a5b4819e93a: Status 404 returned error can't find the container with id b188858963392e946285fe7b0b1160e14cf23e96f0e8d80cff362a5b4819e93a Nov 04 07:34:46.737421 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.737387 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:46.737527 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.737453 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:46.737527 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:46.737482 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:34:47.286312 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:47.286273 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" event={"ID":"2b293d12-35ef-47df-9c77-27f734441df4","Type":"ContainerStarted","Data":"ee8270bfe828eda79d75ab27e52c190d1294cab114932de683a68ac2dcf9ac8b"} Nov 04 07:34:47.286519 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:47.286333 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" event={"ID":"2b293d12-35ef-47df-9c77-27f734441df4","Type":"ContainerStarted","Data":"b188858963392e946285fe7b0b1160e14cf23e96f0e8d80cff362a5b4819e93a"} Nov 04 07:34:47.340795 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:47.340750 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" podStartSLOduration=1.340736404 podStartE2EDuration="1.340736404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:47.338698862 +0000 UTC m=+1122.672638447" watchObservedRunningTime="2026-11-04 07:34:47.340736404 +0000 UTC m=+1122.674675988" Nov 04 07:34:47.446883 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:47.446852 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:47.452218 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:47.452191 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:47.634818 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:47.634786 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:47.806850 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:47.806820 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:47.930016 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:47.929912 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:48.292486 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:48.292398 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:48.293186 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:48.293166 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:34:48.957059 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:48.957015 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:49.182106 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:49.182061 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:49.863699 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:49.863652 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:50.005955 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.005919 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:50.121526 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.119585 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh"] Nov 04 07:34:50.142517 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.134485 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh"] Nov 04 07:34:50.142517 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.134619 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:34:50.223083 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.223014 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-1f6ce4-serving-cert\"" Nov 04 07:34:50.223250 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.223184 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hv4qq\"" Nov 04 07:34:50.223316 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.223247 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Nov 04 07:34:50.262713 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.262660 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:50.306680 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.306647 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh\" (UID: \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:34:50.306850 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.306694 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-metrics-certs\") pod \"maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh\" (UID: \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:34:50.306850 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.306812 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh\" (UID: \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:34:50.306917 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.306854 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npz8v\" (UniqueName: \"kubernetes.io/projected/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-kube-api-access-npz8v\") pod \"maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh\" (UID: \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:34:50.392985 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.392891 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:50.408171 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.408137 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh\" (UID: \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:34:50.408353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.408203 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-metrics-certs\") pod \"maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh\" (UID: \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:34:50.408353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.408262 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh\" (UID: \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:34:50.408353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.408286 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npz8v\" (UniqueName: \"kubernetes.io/projected/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-kube-api-access-npz8v\") pod \"maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh\" (UID: \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:34:50.409011 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.408986 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh\" (UID: \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:34:50.410806 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.410787 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-maas-api-tls\") pod \"maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh\" (UID: \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:34:50.410950 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.410931 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-metrics-certs\") pod \"maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh\" (UID: \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:34:50.431764 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.431703 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npz8v\" (UniqueName: \"kubernetes.io/projected/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-kube-api-access-npz8v\") pod \"maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh\" (UID: \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:34:50.455642 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.455607 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:34:50.554566 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.554528 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:50.738672 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:50.738625 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh"] Nov 04 07:34:50.741447 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:34:50.741416 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20dfe3ac_721e_49c5_8d59_64cf1c9b506f.slice/crio-4cdcfecd1dc2d93e82b73eb89b1c490ce2ce0ea7555f1dcc5b0592b6dc4147b8 WatchSource:0}: Error finding container 4cdcfecd1dc2d93e82b73eb89b1c490ce2ce0ea7555f1dcc5b0592b6dc4147b8: Status 404 returned error can't find the container with id 4cdcfecd1dc2d93e82b73eb89b1c490ce2ce0ea7555f1dcc5b0592b6dc4147b8 Nov 04 07:34:51.309194 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:51.309163 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" event={"ID":"20dfe3ac-721e-49c5-8d59-64cf1c9b506f","Type":"ContainerStarted","Data":"4cdcfecd1dc2d93e82b73eb89b1c490ce2ce0ea7555f1dcc5b0592b6dc4147b8"} Nov 04 07:34:51.433197 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:51.433154 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:51.613515 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:51.613423 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:51.726868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:51.726835 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:52.327617 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.327574 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" event={"ID":"20dfe3ac-721e-49c5-8d59-64cf1c9b506f","Type":"ContainerStarted","Data":"7a066357f9488207d27d9198b35c6cafb838b332645117a3994cfc3579aaa272"} Nov 04 07:34:52.328351 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.328333 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:34:52.354258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.354201 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" podStartSLOduration=1.837934265 podStartE2EDuration="2.35418766s" totalImagesPullingTime="516.253395ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:52.353037671 +0000 UTC m=+1127.686977275" watchObservedRunningTime="2026-11-04 07:34:52.35418766 +0000 UTC m=+1127.688127243" Nov 04 07:34:52.358777 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.358750 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf"] Nov 04 07:34:52.372853 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.372826 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:34:52.383787 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.383755 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-016e6c-serving-cert\"" Nov 04 07:34:52.383947 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.383868 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf"] Nov 04 07:34:52.426879 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.426854 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:52.532103 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.532071 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a52cfeca-0537-4751-8db1-8e1da56f1968-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf\" (UID: \"a52cfeca-0537-4751-8db1-8e1da56f1968\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:34:52.532283 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.532117 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kwc6\" (UniqueName: \"kubernetes.io/projected/a52cfeca-0537-4751-8db1-8e1da56f1968-kube-api-access-5kwc6\") pod \"maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf\" (UID: \"a52cfeca-0537-4751-8db1-8e1da56f1968\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:34:52.532283 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.532213 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a52cfeca-0537-4751-8db1-8e1da56f1968-maas-api-tls\") pod \"maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf\" (UID: \"a52cfeca-0537-4751-8db1-8e1da56f1968\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:34:52.532381 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.532360 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a52cfeca-0537-4751-8db1-8e1da56f1968-metrics-certs\") pod \"maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf\" (UID: \"a52cfeca-0537-4751-8db1-8e1da56f1968\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:34:52.545831 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.545789 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:52.633112 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.633076 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a52cfeca-0537-4751-8db1-8e1da56f1968-maas-api-tls\") pod \"maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf\" (UID: \"a52cfeca-0537-4751-8db1-8e1da56f1968\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:34:52.633283 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.633176 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a52cfeca-0537-4751-8db1-8e1da56f1968-metrics-certs\") pod \"maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf\" (UID: \"a52cfeca-0537-4751-8db1-8e1da56f1968\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:34:52.633283 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.633239 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a52cfeca-0537-4751-8db1-8e1da56f1968-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf\" (UID: \"a52cfeca-0537-4751-8db1-8e1da56f1968\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:34:52.633348 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.633283 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kwc6\" (UniqueName: \"kubernetes.io/projected/a52cfeca-0537-4751-8db1-8e1da56f1968-kube-api-access-5kwc6\") pod \"maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf\" (UID: \"a52cfeca-0537-4751-8db1-8e1da56f1968\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:34:52.633902 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.633877 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a52cfeca-0537-4751-8db1-8e1da56f1968-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf\" (UID: \"a52cfeca-0537-4751-8db1-8e1da56f1968\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:34:52.635714 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.635687 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a52cfeca-0537-4751-8db1-8e1da56f1968-maas-api-tls\") pod \"maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf\" (UID: \"a52cfeca-0537-4751-8db1-8e1da56f1968\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:34:52.635850 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.635831 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a52cfeca-0537-4751-8db1-8e1da56f1968-metrics-certs\") pod \"maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf\" (UID: \"a52cfeca-0537-4751-8db1-8e1da56f1968\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:34:52.646181 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.646153 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kwc6\" (UniqueName: \"kubernetes.io/projected/a52cfeca-0537-4751-8db1-8e1da56f1968-kube-api-access-5kwc6\") pod \"maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf\" (UID: \"a52cfeca-0537-4751-8db1-8e1da56f1968\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:34:52.691673 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.691644 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:34:52.983421 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:52.983396 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf"] Nov 04 07:34:52.985525 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:34:52.985484 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda52cfeca_0537_4751_8db1_8e1da56f1968.slice/crio-d3f6212c2e44c6d570fb98e16d8856c88fb368f0f0c930b7d9f91b0c24060de6 WatchSource:0}: Error finding container d3f6212c2e44c6d570fb98e16d8856c88fb368f0f0c930b7d9f91b0c24060de6: Status 404 returned error can't find the container with id d3f6212c2e44c6d570fb98e16d8856c88fb368f0f0c930b7d9f91b0c24060de6 Nov 04 07:34:53.130838 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.125864 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:53.249344 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.249241 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:53.349742 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.349703 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h"] Nov 04 07:34:53.357656 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.357606 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" event={"ID":"a52cfeca-0537-4751-8db1-8e1da56f1968","Type":"ContainerStarted","Data":"d3f6212c2e44c6d570fb98e16d8856c88fb368f0f0c930b7d9f91b0c24060de6"} Nov 04 07:34:53.357827 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.357747 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:34:53.371731 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.371695 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h"] Nov 04 07:34:53.373430 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.373386 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-60ed4a-serving-cert\"" Nov 04 07:34:53.442677 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.442647 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g65sm\" (UniqueName: \"kubernetes.io/projected/a34590d5-f58f-4073-b0fc-05ae69dc8e66-kube-api-access-g65sm\") pod \"maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h\" (UID: \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:34:53.442831 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.442705 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a34590d5-f58f-4073-b0fc-05ae69dc8e66-maas-api-tls\") pod \"maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h\" (UID: \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:34:53.442831 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.442808 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a34590d5-f58f-4073-b0fc-05ae69dc8e66-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h\" (UID: \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:34:53.442914 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.442902 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a34590d5-f58f-4073-b0fc-05ae69dc8e66-metrics-certs\") pod \"maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h\" (UID: \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:34:53.477169 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.476901 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:53.543584 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.543549 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g65sm\" (UniqueName: \"kubernetes.io/projected/a34590d5-f58f-4073-b0fc-05ae69dc8e66-kube-api-access-g65sm\") pod \"maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h\" (UID: \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:34:53.543742 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.543628 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a34590d5-f58f-4073-b0fc-05ae69dc8e66-maas-api-tls\") pod \"maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h\" (UID: \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:34:53.543742 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.543661 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a34590d5-f58f-4073-b0fc-05ae69dc8e66-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h\" (UID: \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:34:53.543742 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.543706 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a34590d5-f58f-4073-b0fc-05ae69dc8e66-metrics-certs\") pod \"maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h\" (UID: \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:34:53.544334 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.544306 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a34590d5-f58f-4073-b0fc-05ae69dc8e66-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h\" (UID: \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:34:53.546074 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.546050 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a34590d5-f58f-4073-b0fc-05ae69dc8e66-maas-api-tls\") pod \"maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h\" (UID: \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:34:53.546268 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.546241 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a34590d5-f58f-4073-b0fc-05ae69dc8e66-metrics-certs\") pod \"maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h\" (UID: \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:34:53.558866 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.558840 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g65sm\" (UniqueName: \"kubernetes.io/projected/a34590d5-f58f-4073-b0fc-05ae69dc8e66-kube-api-access-g65sm\") pod \"maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h\" (UID: \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:34:53.601526 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.601328 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:53.683068 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.683032 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:34:53.967712 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:53.967679 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h"] Nov 04 07:34:53.972239 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:34:53.972199 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda34590d5_f58f_4073_b0fc_05ae69dc8e66.slice/crio-0eb64664c33709509396a699c427e0244af82e5382adea9dd202f1011d4c38a5 WatchSource:0}: Error finding container 0eb64664c33709509396a699c427e0244af82e5382adea9dd202f1011d4c38a5: Status 404 returned error can't find the container with id 0eb64664c33709509396a699c427e0244af82e5382adea9dd202f1011d4c38a5 Nov 04 07:34:54.018342 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.018302 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:54.141262 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.141226 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:54.368875 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.365949 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:54.372525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.369499 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" event={"ID":"a34590d5-f58f-4073-b0fc-05ae69dc8e66","Type":"ContainerStarted","Data":"0eb64664c33709509396a699c427e0244af82e5382adea9dd202f1011d4c38a5"} Nov 04 07:34:54.380555 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.373047 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" event={"ID":"a52cfeca-0537-4751-8db1-8e1da56f1968","Type":"ContainerStarted","Data":"5893060fe04e2e6aa5a4edc711c498b3077b1e3199d11693cb4068e1e446c14b"} Nov 04 07:34:54.380555 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.373137 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:34:54.395224 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.395108 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" podStartSLOduration=1.934218858 podStartE2EDuration="2.395090972s" totalImagesPullingTime="460.872114ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:54.394481333 +0000 UTC m=+1129.728420982" watchObservedRunningTime="2026-11-04 07:34:54.395090972 +0000 UTC m=+1129.729030556" Nov 04 07:34:54.466529 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.465004 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:54.592147 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.587296 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp"] Nov 04 07:34:54.594451 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.594420 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp"] Nov 04 07:34:54.594595 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.594550 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:34:54.598519 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.598480 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-df3be8-serving-cert\"" Nov 04 07:34:54.599955 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.599917 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:54.666755 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.665938 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp\" (UID: \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:34:54.666755 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.666015 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-maas-api-tls\") pod \"maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp\" (UID: \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:34:54.666755 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.666073 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-metrics-certs\") pod \"maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp\" (UID: \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:34:54.666755 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.666155 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbjsh\" (UniqueName: \"kubernetes.io/projected/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-kube-api-access-fbjsh\") pod \"maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp\" (UID: \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:34:54.708546 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.706306 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:54.772170 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.772113 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-maas-api-tls\") pod \"maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp\" (UID: \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:34:54.772391 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.772207 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-metrics-certs\") pod \"maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp\" (UID: \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:34:54.772391 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.772309 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbjsh\" (UniqueName: \"kubernetes.io/projected/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-kube-api-access-fbjsh\") pod \"maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp\" (UID: \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:34:54.772391 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.772383 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp\" (UID: \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:34:54.773776 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.773325 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp\" (UID: \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:34:54.776968 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.776916 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-maas-api-tls\") pod \"maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp\" (UID: \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:34:54.779530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.779374 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-metrics-certs\") pod \"maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp\" (UID: \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:34:54.786817 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.786794 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbjsh\" (UniqueName: \"kubernetes.io/projected/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-kube-api-access-fbjsh\") pod \"maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp\" (UID: \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:34:54.813855 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.813798 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:54.910971 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:54.910887 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:34:55.207620 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:55.206832 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp"] Nov 04 07:34:55.229866 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:34:55.225860 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d21e3d9_a089_4eac_bbae_1be1ec3fc784.slice/crio-2563aa3f73117be14f507bd82ccb6f4a0aaa97895e316ca2cc241c03b14a345a WatchSource:0}: Error finding container 2563aa3f73117be14f507bd82ccb6f4a0aaa97895e316ca2cc241c03b14a345a: Status 404 returned error can't find the container with id 2563aa3f73117be14f507bd82ccb6f4a0aaa97895e316ca2cc241c03b14a345a Nov 04 07:34:55.396829 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:55.396784 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" event={"ID":"a34590d5-f58f-4073-b0fc-05ae69dc8e66","Type":"ContainerStarted","Data":"ecc71b93856e6f607c2a1e1e0584893463dc953d1f5c645437485907a6392b53"} Nov 04 07:34:55.397234 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:55.396903 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:34:55.399332 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:55.399308 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" event={"ID":"5d21e3d9-a089-4eac-bbae-1be1ec3fc784","Type":"ContainerStarted","Data":"2563aa3f73117be14f507bd82ccb6f4a0aaa97895e316ca2cc241c03b14a345a"} Nov 04 07:34:55.418780 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:55.418731 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" podStartSLOduration=1.8208077409999999 podStartE2EDuration="2.418718978s" totalImagesPullingTime="597.911237ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:55.41800034 +0000 UTC m=+1130.751939932" watchObservedRunningTime="2026-11-04 07:34:55.418718978 +0000 UTC m=+1130.752658561" Nov 04 07:34:55.669531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:55.663175 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:56.407593 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:56.407548 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" event={"ID":"5d21e3d9-a089-4eac-bbae-1be1ec3fc784","Type":"ContainerStarted","Data":"712c6cd3c87be1e18938360082595a80c3c62ef2cdc9a651ee3ab4b12abffe97"} Nov 04 07:34:56.408219 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:56.407636 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:34:56.434400 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:56.434349 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" podStartSLOduration=1.974238126 podStartE2EDuration="2.434334668s" totalImagesPullingTime="460.096542ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:34:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:34:56.433687135 +0000 UTC m=+1131.767626777" watchObservedRunningTime="2026-11-04 07:34:56.434334668 +0000 UTC m=+1131.768274252" Nov 04 07:34:56.472519 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:56.472480 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:57.985531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:57.983988 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:58.160986 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:58.119118 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:59.542690 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:59.485548 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:59.659530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:59.518734 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:34:59.738908 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:59.738853 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:34:59.982852 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:34:59.982796 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:00.968535 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:35:00.959839 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:35:01.234481 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:01.234347 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:01.426437 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:01.425472 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:35:01.427908 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:01.427412 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:35:02.323532 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:02.322734 5385 patch_prober.go:29] interesting pod/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 07:35:02.323532 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:02.322801 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" podUID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 07:35:02.427704 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:02.427666 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:35:02.529214 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:02.529019 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:03.761142 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:03.761103 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:04.884013 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:04.883926 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:04.984874 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:04.984829 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:05.096034 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:05.095735 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:05.811949 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:05.811884 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:06.931048 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:06.931007 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:07.875958 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:07.667993 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:07.875958 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:07.769107 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:08.046121 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:07.937154 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:08.187969 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:08.187879 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:08.357967 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:08.357930 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:08.550769 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:08.550683 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:08.727965 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:08.727933 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:09.472389 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.472354 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:09.682554 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.682498 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:09.732084 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.732053 5385 generic.go:356] "Generic (PLEG): container finished" podID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" containerID="cb66ca294a82a3c4a5dd785ba1077f0b517b1f192067e0556b1896fe8269b2e7" exitCode=0 Nov 04 07:35:09.732262 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.732122 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" event={"ID":"d990bf05-77a4-4ebd-a301-c55f5bb7d526","Type":"ContainerDied","Data":"cb66ca294a82a3c4a5dd785ba1077f0b517b1f192067e0556b1896fe8269b2e7"} Nov 04 07:35:09.732649 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.732624 5385 scope.go:122] "RemoveContainer" containerID="cb66ca294a82a3c4a5dd785ba1077f0b517b1f192067e0556b1896fe8269b2e7" Nov 04 07:35:09.734030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.733992 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:35:09.734143 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.734044 5385 generic.go:356] "Generic (PLEG): container finished" podID="6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" containerID="b1688dc4967fb416014424c6cf890f716885edfa81f525a40b96a3c97cf9cfc8" exitCode=0 Nov 04 07:35:09.734143 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.734104 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:35:09.734143 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.734115 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" event={"ID":"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a","Type":"ContainerDied","Data":"b1688dc4967fb416014424c6cf890f716885edfa81f525a40b96a3c97cf9cfc8"} Nov 04 07:35:09.734364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.734157 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:35:09.734485 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.734467 5385 scope.go:122] "RemoveContainer" containerID="b1688dc4967fb416014424c6cf890f716885edfa81f525a40b96a3c97cf9cfc8" Nov 04 07:35:09.735810 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.735785 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:35:09.735925 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.735850 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:35:09.735925 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.735901 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:35:09.736979 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.736955 5385 generic.go:356] "Generic (PLEG): container finished" podID="6b7e3053-63a4-43c2-ada7-f98132ef6d4d" containerID="05a388de22fde529f354962c82f3dfe918023fd743d50c458d4f22b225143b5f" exitCode=0 Nov 04 07:35:09.737110 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.737089 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" event={"ID":"6b7e3053-63a4-43c2-ada7-f98132ef6d4d","Type":"ContainerDied","Data":"05a388de22fde529f354962c82f3dfe918023fd743d50c458d4f22b225143b5f"} Nov 04 07:35:09.737439 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.737422 5385 scope.go:122] "RemoveContainer" containerID="05a388de22fde529f354962c82f3dfe918023fd743d50c458d4f22b225143b5f" Nov 04 07:35:09.738842 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.738808 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:35:09.738944 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.738923 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:35:09.739003 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.738980 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:35:09.739143 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.739125 5385 generic.go:356] "Generic (PLEG): container finished" podID="2b293d12-35ef-47df-9c77-27f734441df4" containerID="ee8270bfe828eda79d75ab27e52c190d1294cab114932de683a68ac2dcf9ac8b" exitCode=0 Nov 04 07:35:09.739192 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.739172 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" event={"ID":"2b293d12-35ef-47df-9c77-27f734441df4","Type":"ContainerDied","Data":"ee8270bfe828eda79d75ab27e52c190d1294cab114932de683a68ac2dcf9ac8b"} Nov 04 07:35:09.739469 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.739457 5385 scope.go:122] "RemoveContainer" containerID="ee8270bfe828eda79d75ab27e52c190d1294cab114932de683a68ac2dcf9ac8b" Nov 04 07:35:09.743175 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.743147 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:35:09.743279 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.743204 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:35:09.743279 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:09.743237 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:35:10.323478 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:10.323424 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:10.438531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:10.435101 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:10.562288 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:10.562253 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:10.690726 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:10.690673 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:10.844767 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:10.844728 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:10.925906 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:10.925866 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" event={"ID":"2b293d12-35ef-47df-9c77-27f734441df4","Type":"ContainerStarted","Data":"72ec7d8e1bf19d4f7f53cbbe4bbe2d7d77b3d1680c4fbfbf80382ad629b48eaa"} Nov 04 07:35:10.956800 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:10.951586 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" event={"ID":"d990bf05-77a4-4ebd-a301-c55f5bb7d526","Type":"ContainerStarted","Data":"d8b999597a48cd5bac0e08c50029352c92d594bb438f34450efe2f289a6fe8bd"} Nov 04 07:35:10.956800 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:10.951638 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" event={"ID":"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a","Type":"ContainerStarted","Data":"e5b44ae6ca2862b0f5c19ada3b7f909caf3aa7fb55ddaee33ea51cfe3491e1e9"} Nov 04 07:35:10.959087 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:10.959044 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" event={"ID":"6b7e3053-63a4-43c2-ada7-f98132ef6d4d","Type":"ContainerStarted","Data":"eebcb0d628d1dcab4f6f6c9d75597c9ea7361a8bf0e638cf610f53c36d6ac64a"} Nov 04 07:35:11.174559 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:11.162488 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:35:11.174559 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:11.162653 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 07:35:11.174559 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:11.162697 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" podUID="6b7e3053-63a4-43c2-ada7-f98132ef6d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 07:35:11.326245 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:11.324941 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:35:11.326245 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:11.325068 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Nov 04 07:35:11.326245 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:11.325113 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" podUID="6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Nov 04 07:35:11.449771 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:11.449677 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:35:11.450373 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:11.450345 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Nov 04 07:35:11.450578 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:11.450554 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" podUID="2b293d12-35ef-47df-9c77-27f734441df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Nov 04 07:35:11.491529 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:11.490934 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:35:11.491529 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:11.491063 5385 patch_prober.go:29] interesting pod/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Nov 04 07:35:11.491529 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:11.491116 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" podUID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Nov 04 07:35:12.175530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:12.174482 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:12.199532 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:12.179648 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 07:35:12.199532 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:12.179714 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" podUID="6b7e3053-63a4-43c2-ada7-f98132ef6d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 07:35:12.331488 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:12.330980 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Nov 04 07:35:12.331488 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:12.331065 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" podUID="6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Nov 04 07:35:12.450855 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:12.450758 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Nov 04 07:35:12.451146 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:12.451099 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" podUID="2b293d12-35ef-47df-9c77-27f734441df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Nov 04 07:35:12.493084 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:12.493047 5385 patch_prober.go:29] interesting pod/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Nov 04 07:35:12.493410 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:12.493367 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" podUID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Nov 04 07:35:13.011530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:13.008863 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:13.159685 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:13.159648 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 07:35:13.159951 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:13.159922 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" podUID="6b7e3053-63a4-43c2-ada7-f98132ef6d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 07:35:13.323157 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:13.323050 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Nov 04 07:35:13.323851 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:13.323814 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" podUID="6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Nov 04 07:35:13.455117 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:13.455069 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Nov 04 07:35:13.455398 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:13.455365 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" podUID="2b293d12-35ef-47df-9c77-27f734441df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Nov 04 07:35:13.496068 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:13.496033 5385 patch_prober.go:29] interesting pod/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Nov 04 07:35:13.496354 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:13.496320 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" podUID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Nov 04 07:35:13.594853 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:13.594725 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:13.724573 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:13.724050 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:13.838034 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:13.837997 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:14.107778 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:14.107739 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:14.159989 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:14.159957 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 07:35:14.160304 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:14.160262 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" podUID="6b7e3053-63a4-43c2-ada7-f98132ef6d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 07:35:14.329005 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:14.328962 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Nov 04 07:35:14.329638 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:14.329600 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" podUID="6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Nov 04 07:35:14.447972 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:14.447931 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Nov 04 07:35:14.448271 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:14.448242 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" podUID="2b293d12-35ef-47df-9c77-27f734441df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Nov 04 07:35:14.497976 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:14.497938 5385 patch_prober.go:29] interesting pod/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Nov 04 07:35:14.498217 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:14.498192 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" podUID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Nov 04 07:35:15.166531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:15.164979 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 07:35:15.166531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:15.165041 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" podUID="6b7e3053-63a4-43c2-ada7-f98132ef6d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 07:35:15.324541 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:15.322880 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Nov 04 07:35:15.324541 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:15.322942 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" podUID="6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Nov 04 07:35:15.449597 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:15.446726 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Nov 04 07:35:15.449597 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:15.446815 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" podUID="2b293d12-35ef-47df-9c77-27f734441df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Nov 04 07:35:15.502545 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:15.501918 5385 patch_prober.go:29] interesting pod/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Nov 04 07:35:15.502545 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:15.501981 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" podUID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Nov 04 07:35:16.140533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:16.140399 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:16.191087 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:16.191039 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 07:35:16.191283 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:16.191124 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" podUID="6b7e3053-63a4-43c2-ada7-f98132ef6d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 07:35:16.249873 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:16.249837 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:16.282658 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:16.282624 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:35:16.284049 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:16.284022 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:35:16.284576 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:16.284559 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:35:16.326015 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:16.325966 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" start-of-body= Nov 04 07:35:16.326335 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:16.326281 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" podUID="6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.43:15021/healthz/ready\": dial tcp 10.133.0.43:15021: connect: connection refused" Nov 04 07:35:16.394399 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:16.394262 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:16.459553 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:16.455892 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Nov 04 07:35:16.459553 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:16.455958 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" podUID="2b293d12-35ef-47df-9c77-27f734441df4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Nov 04 07:35:16.507543 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:16.506648 5385 patch_prober.go:29] interesting pod/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Nov 04 07:35:16.507543 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:16.506732 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" podUID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Nov 04 07:35:17.172047 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:17.170077 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 07:35:17.172047 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:17.170140 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" podUID="6b7e3053-63a4-43c2-ada7-f98132ef6d4d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 07:35:17.340673 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:17.340573 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:35:17.343247 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:17.343217 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:35:17.618583 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:17.617374 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:17.635185 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:17.635146 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:35:17.636202 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:17.636178 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:35:17.636881 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:17.636855 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:35:17.906978 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:17.906884 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:35:17.908697 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:17.908668 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:35:18.164670 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:18.164593 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:35:18.165683 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:18.165662 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:35:18.270109 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:18.270061 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:18.623808 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:18.623776 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:18.746733 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:18.746704 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:18.908175 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:18.908048 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:19.878621 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:19.878574 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:20.546599 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.546559 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k"] Nov 04 07:35:20.552020 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.551999 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.561871 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.561844 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-60ed4a\"/\"kube-root-ca.crt\"" Nov 04 07:35:20.564055 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.564033 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-60ed4a\"/\"openshift-service-ca.crt\"" Nov 04 07:35:20.568095 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.568069 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k"] Nov 04 07:35:20.568195 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.568187 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-60ed4a\"/\"default-dockercfg-zs5mn\"" Nov 04 07:35:20.568259 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.568246 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-60ed4a\"/\"simulator-w3-60ed4a-kserve-self-signed-certs\"" Nov 04 07:35:20.609128 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.609070 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:20.689341 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.689310 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-dshm\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.689536 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.689456 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-tmp-dir\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.689588 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.689573 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-home\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.689624 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.689610 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85b100f6-ac81-414b-bd68-56276d8cd112-tls-certs\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.689742 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.689716 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqkq6\" (UniqueName: \"kubernetes.io/projected/85b100f6-ac81-414b-bd68-56276d8cd112-kube-api-access-vqkq6\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.689779 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.689764 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-model-cache\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.791488 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.791443 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqkq6\" (UniqueName: \"kubernetes.io/projected/85b100f6-ac81-414b-bd68-56276d8cd112-kube-api-access-vqkq6\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.792287 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.791757 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-model-cache\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.792632 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.792601 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-model-cache\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.792941 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.792849 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-dshm\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.793465 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.793447 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-tmp-dir\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.793944 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.793926 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-home\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.794613 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.794588 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85b100f6-ac81-414b-bd68-56276d8cd112-tls-certs\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.795458 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.794420 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-home\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.795691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.795624 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-tmp-dir\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.806187 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.798656 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:20.806541 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.800402 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85b100f6-ac81-414b-bd68-56276d8cd112-tls-certs\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.806541 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.802201 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-dshm\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.818411 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.818376 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqkq6\" (UniqueName: \"kubernetes.io/projected/85b100f6-ac81-414b-bd68-56276d8cd112-kube-api-access-vqkq6\") pod \"simulator-w3-60ed4a-kserve-b9684f49d-8sm6k\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:20.872723 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:20.872677 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:21.169005 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:21.168967 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k"] Nov 04 07:35:21.172889 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:35:21.172852 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85b100f6_ac81_414b_bd68_56276d8cd112.slice/crio-250dae6b8fb48cf04f2df0736680c42c34ff6e7568d6c2b1e3edc44747f20f8e WatchSource:0}: Error finding container 250dae6b8fb48cf04f2df0736680c42c34ff6e7568d6c2b1e3edc44747f20f8e: Status 404 returned error can't find the container with id 250dae6b8fb48cf04f2df0736680c42c34ff6e7568d6c2b1e3edc44747f20f8e Nov 04 07:35:21.242053 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:21.242025 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" event={"ID":"85b100f6-ac81-414b-bd68-56276d8cd112","Type":"ContainerStarted","Data":"250dae6b8fb48cf04f2df0736680c42c34ff6e7568d6c2b1e3edc44747f20f8e"} Nov 04 07:35:21.783196 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:21.783166 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:22.249026 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:22.248993 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" event={"ID":"85b100f6-ac81-414b-bd68-56276d8cd112","Type":"ContainerStarted","Data":"8ce3de24efd0bf1fa7c2a41e9322243a5027ebe194c671441ec4380339b66d09"} Nov 04 07:35:22.268814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:22.268756 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" podStartSLOduration=2.2687398979999998 podStartE2EDuration="2.268739898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:35:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:35:22.267927815 +0000 UTC m=+1157.601867399" watchObservedRunningTime="2026-11-04 07:35:22.268739898 +0000 UTC m=+1157.602679483" Nov 04 07:35:22.683062 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:22.683029 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:22.873891 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:22.873856 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:23.592034 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:23.591999 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:24.402119 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:24.402085 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:25.077342 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:25.077308 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:25.188354 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:25.188321 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:26.176454 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:26.176421 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:26.979240 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:26.979207 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:27.779968 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:27.779926 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:28.585726 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:28.585697 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:29.404476 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:29.404439 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:30.491646 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:30.491613 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:30.874161 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:30.874132 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:30.876638 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:35:30.876606 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:35:30.887444 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:30.887415 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:31.309488 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:31.309315 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:35:31.497477 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:31.497445 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:32.389336 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:32.389292 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:32.600308 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:32.600274 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:33.282519 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:33.282471 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:34.092992 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:34.092944 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:34.991949 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:34.991915 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:35.628078 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.628043 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l"] Nov 04 07:35:35.632917 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.632894 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.641544 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.641518 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-016e6c\"/\"kube-root-ca.crt\"" Nov 04 07:35:35.641544 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.641527 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-016e6c\"/\"openshift-service-ca.crt\"" Nov 04 07:35:35.642612 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.641625 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-016e6c\"/\"default-dockercfg-dvhvh\"" Nov 04 07:35:35.642612 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.641719 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-016e6c\"/\"premium-w2-016e6c-kserve-self-signed-certs\"" Nov 04 07:35:35.643830 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.643806 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l"] Nov 04 07:35:35.673461 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.673429 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m59w2\" (UniqueName: \"kubernetes.io/projected/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-kube-api-access-m59w2\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.673616 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.673469 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-tmp-dir\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.673675 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.673642 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-dshm\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.673726 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.673700 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-model-cache\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.673832 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.673806 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-home\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.673900 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.673883 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-tls-certs\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.743107 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.743062 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:35.781533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.775183 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-home\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.781533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.775268 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-tls-certs\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.781533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.775390 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m59w2\" (UniqueName: \"kubernetes.io/projected/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-kube-api-access-m59w2\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.781533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.775423 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-tmp-dir\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.781533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.778419 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-dshm\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.781533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.778488 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-model-cache\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.781533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.778989 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-home\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.781533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.780828 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-tmp-dir\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.781533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.781110 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-model-cache\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.787396 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.787363 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-tls-certs\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.799876 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.799840 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-dshm\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.807543 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.807492 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m59w2\" (UniqueName: \"kubernetes.io/projected/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-kube-api-access-m59w2\") pod \"premium-w2-016e6c-kserve-66569fc68c-cxg2l\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.947088 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.946990 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:35.961439 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:35.961408 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:36.244631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:36.244606 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l"] Nov 04 07:35:36.246892 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:35:36.246860 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2411a0ad_44d7_4ff2_9cdd_c53f0f52f2ff.slice/crio-98e8432c24d7b1c7875d1393feb265fd439f4873c49a4b135b858131af9ad760 WatchSource:0}: Error finding container 98e8432c24d7b1c7875d1393feb265fd439f4873c49a4b135b858131af9ad760: Status 404 returned error can't find the container with id 98e8432c24d7b1c7875d1393feb265fd439f4873c49a4b135b858131af9ad760 Nov 04 07:35:36.326538 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:36.326032 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" event={"ID":"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff","Type":"ContainerStarted","Data":"0fd3995a54be0922c32b328411ce66bfcb4be5858534f59d14ad3f8e673b003e"} Nov 04 07:35:36.326538 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:36.326087 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" event={"ID":"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff","Type":"ContainerStarted","Data":"98e8432c24d7b1c7875d1393feb265fd439f4873c49a4b135b858131af9ad760"} Nov 04 07:35:36.346099 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:36.346049 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" podStartSLOduration=1.346036416 podStartE2EDuration="1.346036416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:35:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:35:36.345528553 +0000 UTC m=+1171.679468135" watchObservedRunningTime="2026-11-04 07:35:36.346036416 +0000 UTC m=+1171.679975999" Nov 04 07:35:36.789065 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:36.789025 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:36.947295 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:36.947265 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:37.750515 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:37.750477 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:38.577227 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:38.577180 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:39.586942 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:39.586906 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:40.378897 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:40.378862 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:41.046669 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:41.046635 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:41.515458 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:41.515405 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:42.401490 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:42.401457 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:43.209261 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:43.209230 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:44.169934 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:44.169899 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:45.071344 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:45.071310 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:45.901204 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:45.901170 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:45.949599 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:45.949025 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:45.969152 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:45.969116 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:46.385991 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.385965 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:35:46.530188 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.530145 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks"] Nov 04 07:35:46.535045 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.535016 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.544650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.543608 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks"] Nov 04 07:35:46.544839 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.544817 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-29d3ee\"/\"default-dockercfg-tts4m\"" Nov 04 07:35:46.544975 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.544957 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-29d3ee\"/\"premium-w1-29d3ee-kserve-self-signed-certs\"" Nov 04 07:35:46.576125 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.576094 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-29d3ee\"/\"openshift-service-ca.crt\"" Nov 04 07:35:46.576285 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.576188 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-29d3ee\"/\"kube-root-ca.crt\"" Nov 04 07:35:46.593796 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.593719 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:46.601878 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.601851 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/128cb6a9-354b-4beb-9ea7-eac997fc81b9-tls-certs\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.602018 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.601912 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-model-cache\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.602099 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.602037 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btfjq\" (UniqueName: \"kubernetes.io/projected/128cb6a9-354b-4beb-9ea7-eac997fc81b9-kube-api-access-btfjq\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.602171 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.602123 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-home\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.602233 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.602188 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-dshm\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.602233 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.602213 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-tmp-dir\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.705573 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.703107 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-model-cache\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.705573 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.703184 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btfjq\" (UniqueName: \"kubernetes.io/projected/128cb6a9-354b-4beb-9ea7-eac997fc81b9-kube-api-access-btfjq\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.705573 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.703236 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-home\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.705573 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.703264 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-dshm\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.705573 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.703286 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-tmp-dir\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.705573 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.703372 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/128cb6a9-354b-4beb-9ea7-eac997fc81b9-tls-certs\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.705573 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.704222 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-home\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.705573 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.704482 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-model-cache\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.706547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.706521 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-tmp-dir\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.708041 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.708017 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-dshm\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.731563 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.728828 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btfjq\" (UniqueName: \"kubernetes.io/projected/128cb6a9-354b-4beb-9ea7-eac997fc81b9-kube-api-access-btfjq\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.735070 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.735037 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/128cb6a9-354b-4beb-9ea7-eac997fc81b9-tls-certs\") pod \"premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:46.843514 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.843474 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:46.869919 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:46.869887 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:47.179427 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:47.179401 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks"] Nov 04 07:35:47.181980 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:35:47.181950 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod128cb6a9_354b_4beb_9ea7_eac997fc81b9.slice/crio-dd7aa4d36ccb6bc4552e6438e89d8d7f99153cc75d57555398db5707fe684bbe WatchSource:0}: Error finding container dd7aa4d36ccb6bc4552e6438e89d8d7f99153cc75d57555398db5707fe684bbe: Status 404 returned error can't find the container with id dd7aa4d36ccb6bc4552e6438e89d8d7f99153cc75d57555398db5707fe684bbe Nov 04 07:35:47.381909 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:47.381871 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" event={"ID":"128cb6a9-354b-4beb-9ea7-eac997fc81b9","Type":"ContainerStarted","Data":"74dc4f1cc0dcabb75ebf98701f33acbbda5ef4ae35688d7b4ab589fad0f91962"} Nov 04 07:35:47.382113 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:47.381919 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" event={"ID":"128cb6a9-354b-4beb-9ea7-eac997fc81b9","Type":"ContainerStarted","Data":"dd7aa4d36ccb6bc4552e6438e89d8d7f99153cc75d57555398db5707fe684bbe"} Nov 04 07:35:47.404260 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:47.404208 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" podStartSLOduration=1.4041950060000001 podStartE2EDuration="1.404195006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:35:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:35:47.403056242 +0000 UTC m=+1182.736995828" watchObservedRunningTime="2026-11-04 07:35:47.404195006 +0000 UTC m=+1182.738134589" Nov 04 07:35:47.769856 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:47.769805 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:47.870619 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:47.870583 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:48.112906 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:48.112828 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:48.712461 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:48.712426 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:49.497983 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:49.497949 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:50.306121 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:50.306085 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:51.193656 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:51.193623 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:52.215054 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:52.215005 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:53.293131 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:53.293095 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:54.175466 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:54.175431 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:55.204734 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:55.204698 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:56.130373 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:56.130339 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:56.870517 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:56.870467 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:56.884521 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:56.884476 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:57.017439 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:57.017407 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:57.442988 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:57.442962 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:35:58.081285 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:58.081247 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:58.939445 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:58.939415 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:35:59.784451 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:35:59.784415 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:00.588202 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:00.588166 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:00.876353 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:36:00.876321 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:36:01.394385 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:01.394345 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:02.806528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:02.805783 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:03.974692 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:03.974657 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:04.091152 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:04.091118 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:04.961811 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:04.961774 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:05.770641 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:05.770591 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:06.675564 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:06.675522 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:07.479624 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:07.479590 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:08.278362 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:08.278322 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:09.113327 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.113290 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p"] Nov 04 07:36:09.116266 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.116236 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:09.118236 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.118219 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.138355 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.136321 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-29d3ee-openshift-default-dockercfg-xktsc\"" Nov 04 07:36:09.146710 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.146665 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p"] Nov 04 07:36:09.177905 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.177875 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-workload-socket\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.177905 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.177912 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-envoy\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.178159 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.177944 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-token\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.178159 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.178086 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-credential-socket\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.178159 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.178156 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istiod-ca-cert\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.178289 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.178212 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-workload-certs\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.178289 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.178259 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxklf\" (UniqueName: \"kubernetes.io/projected/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-kube-api-access-mxklf\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.178363 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.178334 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-data\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.178412 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.178381 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-podinfo\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.279868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.279836 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-workload-socket\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.280038 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.279876 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-envoy\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.280038 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.279908 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-token\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.280038 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.279947 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-credential-socket\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.280038 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.279996 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istiod-ca-cert\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.280038 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.280021 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-workload-certs\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.280306 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.280051 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxklf\" (UniqueName: \"kubernetes.io/projected/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-kube-api-access-mxklf\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.280306 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.280082 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-data\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.280306 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.280125 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-podinfo\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.280306 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.280279 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-workload-socket\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.280612 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.280476 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-credential-socket\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.280612 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.280530 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-workload-certs\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.280683 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.280662 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istiod-ca-cert\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.280736 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.280681 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-data\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.282186 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.282162 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-envoy\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.282599 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.282577 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-podinfo\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.292990 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.292961 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-token\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.293093 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.293078 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxklf\" (UniqueName: \"kubernetes.io/projected/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-kube-api-access-mxklf\") pod \"e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.465244 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.465215 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:09.748788 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.748763 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p"] Nov 04 07:36:09.750565 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:36:09.750536 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5db0f011_9979_4ecd_9525_5a02cfd6c8c8.slice/crio-ba52d0ef48ab4848fa3d37dac245bb42c829f5c2c864394b61d6b499b2c69dc6 WatchSource:0}: Error finding container ba52d0ef48ab4848fa3d37dac245bb42c829f5c2c864394b61d6b499b2c69dc6: Status 404 returned error can't find the container with id ba52d0ef48ab4848fa3d37dac245bb42c829f5c2c864394b61d6b499b2c69dc6 Nov 04 07:36:09.752873 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.752841 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:36:09.752947 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.752902 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:36:09.752947 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.752928 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:36:09.985880 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:09.985841 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:10.507239 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:10.507198 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" event={"ID":"5db0f011-9979-4ecd-9525-5a02cfd6c8c8","Type":"ContainerStarted","Data":"0c30904cec1443e4ac89ecc3024848d6230989a4308a3b02c8cb69986f8a80a6"} Nov 04 07:36:10.507731 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:10.507294 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" event={"ID":"5db0f011-9979-4ecd-9525-5a02cfd6c8c8","Type":"ContainerStarted","Data":"ba52d0ef48ab4848fa3d37dac245bb42c829f5c2c864394b61d6b499b2c69dc6"} Nov 04 07:36:10.534016 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:10.533964 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" podStartSLOduration=1.5339489400000001 podStartE2EDuration="1.53394894s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:36:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:36:10.531931838 +0000 UTC m=+1205.865871435" watchObservedRunningTime="2026-11-04 07:36:10.53394894 +0000 UTC m=+1205.867888524" Nov 04 07:36:10.787971 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:10.787877 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:11.470630 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:11.470578 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:11.471819 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:11.471784 5385 patch_prober.go:29] interesting pod/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 07:36:11.471969 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:11.471843 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" podUID="5db0f011-9979-4ecd-9525-5a02cfd6c8c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 07:36:11.725571 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:11.725467 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:12.466021 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:12.465983 5385 patch_prober.go:29] interesting pod/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 07:36:12.466438 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:12.466288 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" podUID="5db0f011-9979-4ecd-9525-5a02cfd6c8c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 07:36:12.688270 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:12.688234 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:13.470361 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:13.469886 5385 patch_prober.go:29] interesting pod/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 07:36:13.470361 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:13.469947 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" podUID="5db0f011-9979-4ecd-9525-5a02cfd6c8c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 07:36:13.829346 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:13.829272 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:14.470972 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:14.470931 5385 patch_prober.go:29] interesting pod/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 07:36:14.471381 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:14.471004 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" podUID="5db0f011-9979-4ecd-9525-5a02cfd6c8c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 07:36:14.971528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:14.971441 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:15.469538 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:15.468851 5385 patch_prober.go:29] interesting pod/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 07:36:15.469538 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:15.468929 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" podUID="5db0f011-9979-4ecd-9525-5a02cfd6c8c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 07:36:16.096555 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:16.096521 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:16.234305 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:16.234269 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:16.465750 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:16.465717 5385 patch_prober.go:29] interesting pod/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Nov 04 07:36:16.465925 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:16.465778 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" podUID="5db0f011-9979-4ecd-9525-5a02cfd6c8c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Nov 04 07:36:16.966936 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:16.966901 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:17.778849 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:17.778813 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:17.994722 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:17.994685 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:17.995099 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:17.995063 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:17.997001 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:17.996979 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:36:18.690693 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:18.690660 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:19.480005 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:19.479967 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:20.281317 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:20.281281 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:21.080900 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:21.080865 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:21.874942 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:21.874903 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:22.673443 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:22.673401 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:23.573366 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:23.573335 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:24.379499 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:24.379465 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:25.252315 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:25.252279 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:26.074363 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:26.074328 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:26.867929 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:26.867898 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:27.672049 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:27.672014 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:28.483472 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:28.483435 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:29.376919 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:29.376883 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:30.103740 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:30.103709 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:30.877378 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:36:30.877332 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:36:30.977716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:30.977661 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:31.885124 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:31.885067 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:32.733605 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:32.733573 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:33.671012 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:33.670978 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:34.474026 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:34.473988 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:35.290163 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:35.290130 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:36.091022 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:36.090989 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:36.982085 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:36.982051 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:37.784896 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:37.784868 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:38.578261 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:38.578229 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:39.379466 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:39.379421 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:40.192184 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.192151 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:40.332919 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.332887 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n"] Nov 04 07:36:40.336683 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.336664 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.344099 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.344072 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-016e6c\"/\"distinct-w2-016e6c-kserve-self-signed-certs\"" Nov 04 07:36:40.348887 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.348864 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n"] Nov 04 07:36:40.375639 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.375603 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20873ddc-3422-4161-8cba-bb4b55fadc04-tls-certs\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.375786 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.375682 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-home\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.375786 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.375719 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-tmp-dir\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.375889 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.375791 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6p4t\" (UniqueName: \"kubernetes.io/projected/20873ddc-3422-4161-8cba-bb4b55fadc04-kube-api-access-j6p4t\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.375950 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.375931 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-dshm\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.376007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.375991 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-model-cache\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.394657 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.394630 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:40.486357 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.486263 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-home\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.487066 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.487040 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-tmp-dir\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.487294 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.487274 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6p4t\" (UniqueName: \"kubernetes.io/projected/20873ddc-3422-4161-8cba-bb4b55fadc04-kube-api-access-j6p4t\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.496535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.487460 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-dshm\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.496849 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.496818 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-model-cache\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.497134 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.497114 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20873ddc-3422-4161-8cba-bb4b55fadc04-tls-certs\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.498168 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.498143 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-tmp-dir\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.504533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.486977 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-home\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.504757 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.503606 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20873ddc-3422-4161-8cba-bb4b55fadc04-tls-certs\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.504882 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.504225 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-dshm\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.508542 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.505341 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-model-cache\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.520398 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.520364 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6p4t\" (UniqueName: \"kubernetes.io/projected/20873ddc-3422-4161-8cba-bb4b55fadc04-kube-api-access-j6p4t\") pod \"distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.651673 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.651645 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:40.931493 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:40.931466 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n"] Nov 04 07:36:40.933813 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:36:40.933779 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20873ddc_3422_4161_8cba_bb4b55fadc04.slice/crio-316386ebb5a20a39442377bf948cc02e5fa28249d372fc5ac3c60962c8112ff1 WatchSource:0}: Error finding container 316386ebb5a20a39442377bf948cc02e5fa28249d372fc5ac3c60962c8112ff1: Status 404 returned error can't find the container with id 316386ebb5a20a39442377bf948cc02e5fa28249d372fc5ac3c60962c8112ff1 Nov 04 07:36:41.107822 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:41.107750 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:41.688566 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:41.688529 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" event={"ID":"20873ddc-3422-4161-8cba-bb4b55fadc04","Type":"ContainerStarted","Data":"da21057a57bf2340d516e4f80856fb1ac06c477744aa9c4c9fc1f3e5a06e7cae"} Nov 04 07:36:41.688566 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:41.688572 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" event={"ID":"20873ddc-3422-4161-8cba-bb4b55fadc04","Type":"ContainerStarted","Data":"316386ebb5a20a39442377bf948cc02e5fa28249d372fc5ac3c60962c8112ff1"} Nov 04 07:36:41.708918 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:41.708860 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" podStartSLOduration=1.708845036 podStartE2EDuration="1.708845036s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:36:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:36:41.707169368 +0000 UTC m=+1237.041108931" watchObservedRunningTime="2026-11-04 07:36:41.708845036 +0000 UTC m=+1237.042784620" Nov 04 07:36:41.871205 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:41.871172 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:42.652481 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:42.652441 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:42.798061 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:42.798026 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:43.772412 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:43.772369 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:44.577659 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:44.577630 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:45.802917 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:45.802874 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:46.655639 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:46.655606 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:47.491218 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:47.491186 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:48.280414 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:48.280376 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:48.613496 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:48.613417 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:48.713464 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:48.713429 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:49.082678 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:49.082642 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:50.074455 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:50.074387 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:50.652488 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:50.652451 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:50.665079 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:50.665051 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:50.755135 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:50.755108 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:36:50.978582 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:50.978475 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:51.466883 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:51.466853 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:51.875953 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:51.875915 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:52.773349 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:52.773316 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:53.759912 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:53.759880 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:54.596137 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:54.596068 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:55.487115 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:55.487055 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:56.304721 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:56.304690 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:57.421793 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:57.421740 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:58.077817 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:58.077783 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:58.273405 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:58.273370 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:58.568496 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:58.568449 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:58.835573 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:58.835456 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:36:59.296565 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:36:59.296500 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:00.090595 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:00.090557 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:00.877107 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:37:00.877060 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:37:01.128750 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:01.128664 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:01.230116 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:01.230082 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:02.110535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:02.110414 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:02.229039 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:02.228994 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:03.158317 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:03.158274 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:03.381664 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:03.381634 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:03.448638 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:03.448548 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh"] Nov 04 07:37:03.452824 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:03.452805 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" Nov 04 07:37:03.462554 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:03.462530 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-6dd194-dockercfg-k44tj\"" Nov 04 07:37:03.462692 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:03.462674 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-6dd194\"" Nov 04 07:37:03.463777 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:03.463752 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh"] Nov 04 07:37:03.495369 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:03.495331 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7946m\" (UniqueName: \"kubernetes.io/projected/b3a8327a-27f1-4947-8567-49372cc916ec-kube-api-access-7946m\") pod \"payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh\" (UID: \"b3a8327a-27f1-4947-8567-49372cc916ec\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" Nov 04 07:37:03.495598 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:03.495573 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3a8327a-27f1-4947-8567-49372cc916ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh\" (UID: \"b3a8327a-27f1-4947-8567-49372cc916ec\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" Nov 04 07:37:03.535161 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:03.535122 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:03.604355 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:03.596934 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3a8327a-27f1-4947-8567-49372cc916ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh\" (UID: \"b3a8327a-27f1-4947-8567-49372cc916ec\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" Nov 04 07:37:03.604355 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:03.597064 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7946m\" (UniqueName: \"kubernetes.io/projected/b3a8327a-27f1-4947-8567-49372cc916ec-kube-api-access-7946m\") pod \"payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh\" (UID: \"b3a8327a-27f1-4947-8567-49372cc916ec\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" Nov 04 07:37:03.604355 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:03.602046 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3a8327a-27f1-4947-8567-49372cc916ec-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh\" (UID: \"b3a8327a-27f1-4947-8567-49372cc916ec\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" Nov 04 07:37:03.619680 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:03.619645 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7946m\" (UniqueName: \"kubernetes.io/projected/b3a8327a-27f1-4947-8567-49372cc916ec-kube-api-access-7946m\") pod \"payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh\" (UID: \"b3a8327a-27f1-4947-8567-49372cc916ec\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" Nov 04 07:37:03.767290 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:03.767201 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" Nov 04 07:37:04.044621 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:04.044589 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh"] Nov 04 07:37:04.046684 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:37:04.046646 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3a8327a_27f1_4947_8567_49372cc916ec.slice/crio-e24126d69fc216ebec367ac2e10631fbf462afc67d4d060d92f18cc3bdfdb56f WatchSource:0}: Error finding container e24126d69fc216ebec367ac2e10631fbf462afc67d4d060d92f18cc3bdfdb56f: Status 404 returned error can't find the container with id e24126d69fc216ebec367ac2e10631fbf462afc67d4d060d92f18cc3bdfdb56f Nov 04 07:37:04.198693 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:04.198661 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:04.818106 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:04.818074 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" event={"ID":"b3a8327a-27f1-4947-8567-49372cc916ec","Type":"ContainerStarted","Data":"4b57ad09d8630f4510526b60e1e3c01a9391a9347dfe9de7335831b55d9072e3"} Nov 04 07:37:04.818551 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:04.818128 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" Nov 04 07:37:04.818551 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:04.818152 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" event={"ID":"b3a8327a-27f1-4947-8567-49372cc916ec","Type":"ContainerStarted","Data":"e24126d69fc216ebec367ac2e10631fbf462afc67d4d060d92f18cc3bdfdb56f"} Nov 04 07:37:04.837558 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:04.837482 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" podStartSLOduration=1.837468602 podStartE2EDuration="1.837468602s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:04.835787689 +0000 UTC m=+1260.169727293" watchObservedRunningTime="2026-11-04 07:37:04.837468602 +0000 UTC m=+1260.171408186" Nov 04 07:37:05.103981 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:05.103899 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:06.277714 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:06.277676 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:07.036467 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.036428 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x"] Nov 04 07:37:07.041034 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.041008 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:37:07.051959 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.051932 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-6dd194-serving-cert\"" Nov 04 07:37:07.055023 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.054999 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x"] Nov 04 07:37:07.089367 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.089328 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:07.142150 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.142119 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-metrics-certs\") pod \"maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x\" (UID: \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:37:07.142337 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.142272 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-maas-api-tls\") pod \"maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x\" (UID: \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:37:07.142454 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.142418 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s67wh\" (UniqueName: \"kubernetes.io/projected/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-kube-api-access-s67wh\") pod \"maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x\" (UID: \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:37:07.142709 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.142689 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x\" (UID: \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:37:07.189448 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.189411 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:07.243777 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.243745 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-maas-api-tls\") pod \"maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x\" (UID: \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:37:07.243952 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.243828 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s67wh\" (UniqueName: \"kubernetes.io/projected/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-kube-api-access-s67wh\") pod \"maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x\" (UID: \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:37:07.243952 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.243878 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x\" (UID: \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:37:07.243952 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:37:07.243894 5385 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-serving-cert: secret "maas-api-e2e-extmod-a-6dd194-serving-cert" not found Nov 04 07:37:07.243952 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.243930 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-metrics-certs\") pod \"maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x\" (UID: \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:37:07.244165 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:37:07.243958 5385 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-maas-api-tls podName:f6e3296f-cbf9-43d0-8a3f-86e6d0190b63 nodeName:}" failed. No retries permitted until 2026-11-04 07:37:07.743940011 +0000 UTC m=+1263.077879573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-maas-api-tls") pod "maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" (UID: "f6e3296f-cbf9-43d0-8a3f-86e6d0190b63") : secret "maas-api-e2e-extmod-a-6dd194-serving-cert" not found Nov 04 07:37:07.244638 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.244614 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x\" (UID: \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:37:07.246380 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.246358 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-metrics-certs\") pod \"maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x\" (UID: \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:37:07.257161 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.257133 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s67wh\" (UniqueName: \"kubernetes.io/projected/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-kube-api-access-s67wh\") pod \"maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x\" (UID: \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:37:07.452565 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.452532 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:07.749543 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.749431 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-maas-api-tls\") pod \"maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x\" (UID: \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:37:07.752096 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.752063 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-maas-api-tls\") pod \"maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x\" (UID: \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:37:07.955235 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:07.955183 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:37:08.064795 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:08.064766 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:08.241642 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:08.241610 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x"] Nov 04 07:37:08.242835 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:37:08.242806 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6e3296f_cbf9_43d0_8a3f_86e6d0190b63.slice/crio-43e7b850a33ecb75f15406b8de4f6a84507454821427b9cc2102bc2258a7480a WatchSource:0}: Error finding container 43e7b850a33ecb75f15406b8de4f6a84507454821427b9cc2102bc2258a7480a: Status 404 returned error can't find the container with id 43e7b850a33ecb75f15406b8de4f6a84507454821427b9cc2102bc2258a7480a Nov 04 07:37:08.244289 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:08.244269 5385 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 07:37:08.871806 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:08.871493 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" event={"ID":"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63","Type":"ContainerStarted","Data":"9273cdd09d306a9bf1d580fdd8c00f8273fc318006fc8e5404d7e94e54b18111"} Nov 04 07:37:08.871806 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:08.871592 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:37:08.871806 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:08.871612 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" event={"ID":"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63","Type":"ContainerStarted","Data":"43e7b850a33ecb75f15406b8de4f6a84507454821427b9cc2102bc2258a7480a"} Nov 04 07:37:08.895607 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:08.895541 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" podStartSLOduration=1.430266541 podStartE2EDuration="1.895521227s" totalImagesPullingTime="465.254686ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:08.893307835 +0000 UTC m=+1264.227247421" watchObservedRunningTime="2026-11-04 07:37:08.895521227 +0000 UTC m=+1264.229460806" Nov 04 07:37:08.986816 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:08.986732 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:09.793476 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:09.793445 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:10.005435 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.005397 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g"] Nov 04 07:37:10.009259 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.009234 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.024747 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.024721 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-016e6c\"/\"distinct-2-w2-016e6c-kserve-self-signed-certs\"" Nov 04 07:37:10.027429 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.027404 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g"] Nov 04 07:37:10.086435 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.086350 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:10.089683 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.089661 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-home\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.089756 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.089705 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-tmp-dir\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.089756 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.089725 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-dshm\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.089870 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.089857 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-model-cache\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.089927 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.089899 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7vhm\" (UniqueName: \"kubernetes.io/projected/c2b40c45-d563-4994-876c-b150d0403dd6-kube-api-access-t7vhm\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.090009 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.089993 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2b40c45-d563-4994-876c-b150d0403dd6-tls-certs\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.197549 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.197115 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-tmp-dir\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.197549 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.197167 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-dshm\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.197549 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.197275 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-model-cache\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.197549 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.197340 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7vhm\" (UniqueName: \"kubernetes.io/projected/c2b40c45-d563-4994-876c-b150d0403dd6-kube-api-access-t7vhm\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.197549 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.197453 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2b40c45-d563-4994-876c-b150d0403dd6-tls-certs\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.197549 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.197520 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-home\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.198018 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.197997 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-home\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.201536 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.198271 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-tmp-dir\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.201536 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.199181 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-model-cache\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.209525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.205888 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-dshm\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.227692 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.216249 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2b40c45-d563-4994-876c-b150d0403dd6-tls-certs\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.227692 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.223265 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7vhm\" (UniqueName: \"kubernetes.io/projected/c2b40c45-d563-4994-876c-b150d0403dd6-kube-api-access-t7vhm\") pod \"distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.321663 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.321633 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:10.600146 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.600121 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g"] Nov 04 07:37:10.601781 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:37:10.601742 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2b40c45_d563_4994_876c_b150d0403dd6.slice/crio-201c6fea2a61244d79d629e79f263c9609fc2d5fd4a2a7f8027e09d805dafff1 WatchSource:0}: Error finding container 201c6fea2a61244d79d629e79f263c9609fc2d5fd4a2a7f8027e09d805dafff1: Status 404 returned error can't find the container with id 201c6fea2a61244d79d629e79f263c9609fc2d5fd4a2a7f8027e09d805dafff1 Nov 04 07:37:10.680397 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.680370 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:10.882208 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.882111 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" event={"ID":"c2b40c45-d563-4994-876c-b150d0403dd6","Type":"ContainerStarted","Data":"600e32d5b2ee156ba580e749e9703769dee45938a0e0e4be8450150c5675e24a"} Nov 04 07:37:10.882208 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.882157 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" event={"ID":"c2b40c45-d563-4994-876c-b150d0403dd6","Type":"ContainerStarted","Data":"201c6fea2a61244d79d629e79f263c9609fc2d5fd4a2a7f8027e09d805dafff1"} Nov 04 07:37:10.902870 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:10.902822 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" podStartSLOduration=1.902800256 podStartE2EDuration="1.902800256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:10.9009715 +0000 UTC m=+1266.234911081" watchObservedRunningTime="2026-11-04 07:37:10.902800256 +0000 UTC m=+1266.236739888" Nov 04 07:37:11.322016 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:11.321928 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:11.583051 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:11.582969 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:12.490931 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:12.490897 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:13.386077 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:13.386043 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:14.204756 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:14.204718 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:14.532826 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:14.532744 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:14.881348 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:14.881321 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:37:15.195344 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:15.195261 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:15.827247 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:15.827217 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" Nov 04 07:37:16.191943 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:16.191893 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:16.291387 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:16.291359 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:17.150251 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:17.150199 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:18.294251 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:18.294195 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:19.098007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:19.097976 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:19.995065 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:19.995028 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:20.322449 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:20.322359 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:20.335243 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:20.335214 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:20.948324 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:20.948296 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:37:20.975961 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:20.975932 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:22.018865 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:22.018836 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:22.132908 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:22.132869 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:22.972616 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:22.972574 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:23.452821 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.452784 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx"] Nov 04 07:37:23.455963 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.455923 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:23.456963 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.456942 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.467255 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.467229 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-60ed4a\"/\"distinct-2-w3-60ed4a-kserve-self-signed-certs\"" Nov 04 07:37:23.471325 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.471299 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx"] Nov 04 07:37:23.556849 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.556813 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:23.568948 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.568916 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sp6cq\" (UniqueName: \"kubernetes.io/projected/f4b66a64-9637-4b05-a20e-880ad2913b66-kube-api-access-sp6cq\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.569066 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.568953 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-dshm\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.569120 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.569107 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4b66a64-9637-4b05-a20e-880ad2913b66-tls-certs\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.569157 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.569138 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-model-cache\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.569191 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.569161 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-home\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.569292 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.569270 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-tmp-dir\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.670369 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.670323 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4b66a64-9637-4b05-a20e-880ad2913b66-tls-certs\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.670591 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.670397 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-model-cache\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.670591 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.670439 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-home\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.670591 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.670486 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-tmp-dir\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.670783 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.670610 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sp6cq\" (UniqueName: \"kubernetes.io/projected/f4b66a64-9637-4b05-a20e-880ad2913b66-kube-api-access-sp6cq\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.670783 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.670644 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-dshm\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.671810 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.671345 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-home\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.671942 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.671808 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-tmp-dir\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.672071 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.672043 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-model-cache\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.689716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.688412 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-dshm\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.689716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.689070 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4b66a64-9637-4b05-a20e-880ad2913b66-tls-certs\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.689716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.689101 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sp6cq\" (UniqueName: \"kubernetes.io/projected/f4b66a64-9637-4b05-a20e-880ad2913b66-kube-api-access-sp6cq\") pod \"distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.775406 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.775335 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:23.913018 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:23.912989 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:24.069670 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:24.069640 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx"] Nov 04 07:37:24.071300 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:37:24.071268 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4b66a64_9637_4b05_a20e_880ad2913b66.slice/crio-441c79a4075d8be2adbc36609fd6b32a3e3c702937174cad6dc82e9d20020fc2 WatchSource:0}: Error finding container 441c79a4075d8be2adbc36609fd6b32a3e3c702937174cad6dc82e9d20020fc2: Status 404 returned error can't find the container with id 441c79a4075d8be2adbc36609fd6b32a3e3c702937174cad6dc82e9d20020fc2 Nov 04 07:37:24.124452 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:24.124420 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:24.222153 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:24.222118 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:24.955475 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:24.955440 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:25.024533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:25.023584 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" event={"ID":"f4b66a64-9637-4b05-a20e-880ad2913b66","Type":"ContainerStarted","Data":"91d05ad771bb658b6e4934cd24a8ccc92f332bbd998ca9487f6dfa06bebefa36"} Nov 04 07:37:25.024533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:25.023659 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" event={"ID":"f4b66a64-9637-4b05-a20e-880ad2913b66","Type":"ContainerStarted","Data":"441c79a4075d8be2adbc36609fd6b32a3e3c702937174cad6dc82e9d20020fc2"} Nov 04 07:37:25.053136 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:25.053084 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" podStartSLOduration=2.053071927 podStartE2EDuration="2.053071927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:25.051526596 +0000 UTC m=+1280.385466181" watchObservedRunningTime="2026-11-04 07:37:25.053071927 +0000 UTC m=+1280.387011510" Nov 04 07:37:25.775605 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:25.775574 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:25.786835 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:25.786800 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:26.819924 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:26.819889 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:27.502882 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:27.502844 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:27.604691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:27.604654 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:27.918156 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:27.918122 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:28.688258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:28.688226 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:28.725593 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:28.725563 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5"] Nov 04 07:37:28.729474 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:28.729456 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" Nov 04 07:37:28.738917 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:28.738894 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-812032\"" Nov 04 07:37:28.741640 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:28.741613 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5"] Nov 04 07:37:28.813390 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:28.813362 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:28.834714 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:28.834679 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fnzt\" (UniqueName: \"kubernetes.io/projected/947a11da-cef0-4a9c-8e51-568588b5885a-kube-api-access-6fnzt\") pod \"payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5\" (UID: \"947a11da-cef0-4a9c-8e51-568588b5885a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" Nov 04 07:37:28.834872 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:28.834805 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/947a11da-cef0-4a9c-8e51-568588b5885a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5\" (UID: \"947a11da-cef0-4a9c-8e51-568588b5885a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" Nov 04 07:37:28.929530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:28.921967 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:28.941734 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:28.941627 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fnzt\" (UniqueName: \"kubernetes.io/projected/947a11da-cef0-4a9c-8e51-568588b5885a-kube-api-access-6fnzt\") pod \"payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5\" (UID: \"947a11da-cef0-4a9c-8e51-568588b5885a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" Nov 04 07:37:28.941892 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:28.941823 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/947a11da-cef0-4a9c-8e51-568588b5885a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5\" (UID: \"947a11da-cef0-4a9c-8e51-568588b5885a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" Nov 04 07:37:28.945039 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:28.944982 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/947a11da-cef0-4a9c-8e51-568588b5885a-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5\" (UID: \"947a11da-cef0-4a9c-8e51-568588b5885a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" Nov 04 07:37:28.955786 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:28.955753 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fnzt\" (UniqueName: \"kubernetes.io/projected/947a11da-cef0-4a9c-8e51-568588b5885a-kube-api-access-6fnzt\") pod \"payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5\" (UID: \"947a11da-cef0-4a9c-8e51-568588b5885a\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" Nov 04 07:37:29.051959 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.051924 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" Nov 04 07:37:29.346853 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.346823 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8"] Nov 04 07:37:29.349041 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:37:29.349007 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod947a11da_cef0_4a9c_8e51_568588b5885a.slice/crio-6e6b8f23bad13cf33e3ab13b6bb0fcacca30e1d922c4aa9cd59facb98b3e9a3c WatchSource:0}: Error finding container 6e6b8f23bad13cf33e3ab13b6bb0fcacca30e1d922c4aa9cd59facb98b3e9a3c: Status 404 returned error can't find the container with id 6e6b8f23bad13cf33e3ab13b6bb0fcacca30e1d922c4aa9cd59facb98b3e9a3c Nov 04 07:37:29.351921 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.351902 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5"] Nov 04 07:37:29.352008 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.351988 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.365605 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.365580 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8"] Nov 04 07:37:29.448189 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.448156 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.448317 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.448211 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.448317 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.448301 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.448402 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.448345 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.448439 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.448412 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.448568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.448547 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.448612 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.448592 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.448682 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.448668 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.448719 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.448698 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt6w7\" (UniqueName: \"kubernetes.io/projected/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-kube-api-access-xt6w7\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.549790 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.549757 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.549952 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.549813 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.549952 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.549843 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.549952 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.549867 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.549952 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.549889 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt6w7\" (UniqueName: \"kubernetes.io/projected/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-kube-api-access-xt6w7\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.550168 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.549959 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.550168 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.549978 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.550168 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.550000 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.550168 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.550036 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.550381 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.550212 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.550381 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.550248 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.550381 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.550315 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.550474 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.550386 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.550826 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.550800 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.552291 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.552269 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.552432 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.552414 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.562232 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.562212 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.562378 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.562359 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt6w7\" (UniqueName: \"kubernetes.io/projected/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-kube-api-access-xt6w7\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.687353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.687327 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:29.988783 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:29.988702 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:30.001276 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:30.001249 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8"] Nov 04 07:37:30.002402 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:37:30.002371 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e2400ce_b1f1_48a9_b0bb_be239c8dd0ce.slice/crio-43163122b7b39a53a976cec71982e93a41ba84176364c345b88b632b4490d7fb WatchSource:0}: Error finding container 43163122b7b39a53a976cec71982e93a41ba84176364c345b88b632b4490d7fb: Status 404 returned error can't find the container with id 43163122b7b39a53a976cec71982e93a41ba84176364c345b88b632b4490d7fb Nov 04 07:37:30.004840 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:30.004812 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:37:30.004929 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:30.004870 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:37:30.004929 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:30.004899 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:37:30.080760 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:30.080725 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" event={"ID":"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce","Type":"ContainerStarted","Data":"43163122b7b39a53a976cec71982e93a41ba84176364c345b88b632b4490d7fb"} Nov 04 07:37:30.083158 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:30.083122 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" event={"ID":"947a11da-cef0-4a9c-8e51-568588b5885a","Type":"ContainerStarted","Data":"26baebbe1d2e92d97dca787d9fa3f88f8e8c3cbd6063a6daf985f224bb51177b"} Nov 04 07:37:30.083358 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:30.083190 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" event={"ID":"947a11da-cef0-4a9c-8e51-568588b5885a","Type":"ContainerStarted","Data":"6e6b8f23bad13cf33e3ab13b6bb0fcacca30e1d922c4aa9cd59facb98b3e9a3c"} Nov 04 07:37:30.083358 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:30.083240 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" Nov 04 07:37:30.112691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:30.112651 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" podStartSLOduration=2.112636401 podStartE2EDuration="2.112636401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:30.108479879 +0000 UTC m=+1285.442419465" watchObservedRunningTime="2026-11-04 07:37:30.112636401 +0000 UTC m=+1285.446575989" Nov 04 07:37:30.899304 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:37:30.899250 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:37:30.899537 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:30.899493 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:31.091410 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:31.091364 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" event={"ID":"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce","Type":"ContainerStarted","Data":"7f974ff8777c22011a57274212e93046e281fac8dc300aeba771035aa6890817"} Nov 04 07:37:31.117361 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:31.117300 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" podStartSLOduration=2.117287235 podStartE2EDuration="2.117287235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:31.115637839 +0000 UTC m=+1286.449577422" watchObservedRunningTime="2026-11-04 07:37:31.117287235 +0000 UTC m=+1286.451226819" Nov 04 07:37:31.688033 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:31.687987 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:31.689722 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:31.689693 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Nov 04 07:37:31.689848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:31.689744 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" podUID="7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Nov 04 07:37:31.798626 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:31.798586 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:32.687923 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:32.687890 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Nov 04 07:37:32.688311 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:32.687940 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" podUID="7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Nov 04 07:37:32.694271 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:32.694246 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:33.615481 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:33.615445 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:33.689442 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:33.687816 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Nov 04 07:37:33.689442 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:33.687876 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" podUID="7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Nov 04 07:37:33.716253 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:33.716220 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:33.776371 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:33.776339 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:33.789779 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:33.789751 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:33.812891 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:33.812859 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:34.190183 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:34.190155 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:37:34.589022 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:34.588942 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:34.691531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:34.688020 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Nov 04 07:37:34.691531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:34.688099 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" podUID="7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Nov 04 07:37:34.712810 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:34.712767 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:34.812836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:34.812799 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:35.143678 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:35.143622 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:35.275701 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:35.275654 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:35.454026 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:35.453884 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:35.671116 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:35.671072 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:35.688321 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:35.688291 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Nov 04 07:37:35.688447 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:35.688351 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" podUID="7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Nov 04 07:37:35.770585 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:35.769890 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:36.577795 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:36.577754 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:36.694306 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:36.694259 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Nov 04 07:37:36.694497 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:36.694336 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" podUID="7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Nov 04 07:37:37.611718 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:37.611684 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:37.693332 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:37.693297 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:37.693738 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:37.693685 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:37.694827 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:37.694807 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:37:38.783486 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:38.783447 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:38.887342 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:38.887260 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:39.009361 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.009319 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:39.162492 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.162458 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:39.320308 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.320274 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd"] Nov 04 07:37:39.351039 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.351000 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:39.364202 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.364173 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd"] Nov 04 07:37:39.364628 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.364558 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:37:39.383533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.379683 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-n5s78\"" Nov 04 07:37:39.383533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.379759 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-d8afd72246fc\"" Nov 04 07:37:39.478580 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.433860 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\"" Nov 04 07:37:39.479080 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.445668 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\"" Nov 04 07:37:39.479355 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.445738 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\"" Nov 04 07:37:39.479449 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.445845 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\"" Nov 04 07:37:39.587973 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.586252 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdbpf\" (UniqueName: \"kubernetes.io/projected/87e99792-812e-4038-8262-13eaf2a52347-kube-api-access-bdbpf\") pod \"payload-processing-external-model-646f5f8dd4-s2qzd\" (UID: \"87e99792-812e-4038-8262-13eaf2a52347\") " pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:37:39.587973 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.586406 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/87e99792-812e-4038-8262-13eaf2a52347-routing-overlay-volume\") pod \"payload-processing-external-model-646f5f8dd4-s2qzd\" (UID: \"87e99792-812e-4038-8262-13eaf2a52347\") " pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:37:39.598929 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.598889 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/87e99792-812e-4038-8262-13eaf2a52347-provider-credentials\") pod \"payload-processing-external-model-646f5f8dd4-s2qzd\" (UID: \"87e99792-812e-4038-8262-13eaf2a52347\") " pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:37:39.599338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.599308 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87e99792-812e-4038-8262-13eaf2a52347-plugins-config-volume\") pod \"payload-processing-external-model-646f5f8dd4-s2qzd\" (UID: \"87e99792-812e-4038-8262-13eaf2a52347\") " pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:37:39.701075 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.701029 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87e99792-812e-4038-8262-13eaf2a52347-plugins-config-volume\") pod \"payload-processing-external-model-646f5f8dd4-s2qzd\" (UID: \"87e99792-812e-4038-8262-13eaf2a52347\") " pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:37:39.701412 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.701383 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdbpf\" (UniqueName: \"kubernetes.io/projected/87e99792-812e-4038-8262-13eaf2a52347-kube-api-access-bdbpf\") pod \"payload-processing-external-model-646f5f8dd4-s2qzd\" (UID: \"87e99792-812e-4038-8262-13eaf2a52347\") " pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:37:39.701572 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.701485 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/87e99792-812e-4038-8262-13eaf2a52347-routing-overlay-volume\") pod \"payload-processing-external-model-646f5f8dd4-s2qzd\" (UID: \"87e99792-812e-4038-8262-13eaf2a52347\") " pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:37:39.701638 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.701591 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/87e99792-812e-4038-8262-13eaf2a52347-provider-credentials\") pod \"payload-processing-external-model-646f5f8dd4-s2qzd\" (UID: \"87e99792-812e-4038-8262-13eaf2a52347\") " pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:37:39.701958 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.701932 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87e99792-812e-4038-8262-13eaf2a52347-plugins-config-volume\") pod \"payload-processing-external-model-646f5f8dd4-s2qzd\" (UID: \"87e99792-812e-4038-8262-13eaf2a52347\") " pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:37:39.702801 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.702768 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/87e99792-812e-4038-8262-13eaf2a52347-routing-overlay-volume\") pod \"payload-processing-external-model-646f5f8dd4-s2qzd\" (UID: \"87e99792-812e-4038-8262-13eaf2a52347\") " pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:37:39.704396 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.704370 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/87e99792-812e-4038-8262-13eaf2a52347-provider-credentials\") pod \"payload-processing-external-model-646f5f8dd4-s2qzd\" (UID: \"87e99792-812e-4038-8262-13eaf2a52347\") " pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:37:39.723615 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.723582 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdbpf\" (UniqueName: \"kubernetes.io/projected/87e99792-812e-4038-8262-13eaf2a52347-kube-api-access-bdbpf\") pod \"payload-processing-external-model-646f5f8dd4-s2qzd\" (UID: \"87e99792-812e-4038-8262-13eaf2a52347\") " pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:37:39.787640 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:39.787533 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:37:40.106159 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.106134 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd"] Nov 04 07:37:40.109318 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:37:40.109284 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87e99792_812e_4038_8262_13eaf2a52347.slice/crio-e9d4d3bf0e5e2e7e0cefcda2232f2a4f1b8258b9d50fec8b443ecfb8333acf94 WatchSource:0}: Error finding container e9d4d3bf0e5e2e7e0cefcda2232f2a4f1b8258b9d50fec8b443ecfb8333acf94: Status 404 returned error can't find the container with id e9d4d3bf0e5e2e7e0cefcda2232f2a4f1b8258b9d50fec8b443ecfb8333acf94 Nov 04 07:37:40.220653 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.220615 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" event={"ID":"87e99792-812e-4038-8262-13eaf2a52347","Type":"ContainerStarted","Data":"46b14ffbe2fd6231f38019d565febbdeb6c0c380b3c6bce10525f4b4265f102a"} Nov 04 07:37:40.220899 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.220877 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" event={"ID":"87e99792-812e-4038-8262-13eaf2a52347","Type":"ContainerStarted","Data":"e9d4d3bf0e5e2e7e0cefcda2232f2a4f1b8258b9d50fec8b443ecfb8333acf94"} Nov 04 07:37:40.222060 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.222040 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:37:40.256876 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.255122 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" podStartSLOduration=1.255103221 podStartE2EDuration="1.255103221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:40.249317356 +0000 UTC m=+1295.583256941" watchObservedRunningTime="2026-11-04 07:37:40.255103221 +0000 UTC m=+1295.589042787" Nov 04 07:37:40.267133 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.267101 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:40.306214 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.306180 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt"] Nov 04 07:37:40.310792 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.310769 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" Nov 04 07:37:40.315002 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.314971 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-2a8c48\"" Nov 04 07:37:40.324705 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.324659 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt"] Nov 04 07:37:40.411058 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.411027 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrjmp\" (UniqueName: \"kubernetes.io/projected/5c14cd7f-ad7e-446c-be34-1b251628c970-kube-api-access-nrjmp\") pod \"payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt\" (UID: \"5c14cd7f-ad7e-446c-be34-1b251628c970\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" Nov 04 07:37:40.411256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.411098 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c14cd7f-ad7e-446c-be34-1b251628c970-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt\" (UID: \"5c14cd7f-ad7e-446c-be34-1b251628c970\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" Nov 04 07:37:40.413967 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.413901 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:40.521015 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.520952 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:40.522010 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.521978 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrjmp\" (UniqueName: \"kubernetes.io/projected/5c14cd7f-ad7e-446c-be34-1b251628c970-kube-api-access-nrjmp\") pod \"payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt\" (UID: \"5c14cd7f-ad7e-446c-be34-1b251628c970\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" Nov 04 07:37:40.522562 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.522540 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c14cd7f-ad7e-446c-be34-1b251628c970-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt\" (UID: \"5c14cd7f-ad7e-446c-be34-1b251628c970\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" Nov 04 07:37:40.523652 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.523632 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c14cd7f-ad7e-446c-be34-1b251628c970-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt\" (UID: \"5c14cd7f-ad7e-446c-be34-1b251628c970\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" Nov 04 07:37:40.550321 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.550285 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrjmp\" (UniqueName: \"kubernetes.io/projected/5c14cd7f-ad7e-446c-be34-1b251628c970-kube-api-access-nrjmp\") pod \"payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt\" (UID: \"5c14cd7f-ad7e-446c-be34-1b251628c970\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" Nov 04 07:37:40.626362 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.626326 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" Nov 04 07:37:40.931907 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:40.931875 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt"] Nov 04 07:37:40.932979 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:37:40.932944 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c14cd7f_ad7e_446c_be34_1b251628c970.slice/crio-c86d2da4a6647ae563cd65d19b97f7b8e6535bea26ab698f1271cad2b6181d1a WatchSource:0}: Error finding container c86d2da4a6647ae563cd65d19b97f7b8e6535bea26ab698f1271cad2b6181d1a: Status 404 returned error can't find the container with id c86d2da4a6647ae563cd65d19b97f7b8e6535bea26ab698f1271cad2b6181d1a Nov 04 07:37:41.089571 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.069157 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:41.122699 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.122669 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" Nov 04 07:37:41.129175 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.129147 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8"] Nov 04 07:37:41.133613 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.133587 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" Nov 04 07:37:41.150827 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.145956 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8"] Nov 04 07:37:41.175780 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.175751 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf"] Nov 04 07:37:41.187526 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.155332 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-3da894\"" Nov 04 07:37:41.198554 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.194696 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:41.207529 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.203995 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf"] Nov 04 07:37:41.207529 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.204108 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" Nov 04 07:37:41.230910 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.230825 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-3da894-dockercfg-dgdbm\"" Nov 04 07:37:41.231392 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.231369 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a306921b-4523-4b47-915c-2b6251c276fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf\" (UID: \"a306921b-4523-4b47-915c-2b6251c276fc\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" Nov 04 07:37:41.231534 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.231411 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80160402-c66f-43db-8fd4-91fc30fa1460-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8\" (UID: \"80160402-c66f-43db-8fd4-91fc30fa1460\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" Nov 04 07:37:41.231534 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.231448 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qstlq\" (UniqueName: \"kubernetes.io/projected/80160402-c66f-43db-8fd4-91fc30fa1460-kube-api-access-qstlq\") pod \"payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8\" (UID: \"80160402-c66f-43db-8fd4-91fc30fa1460\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" Nov 04 07:37:41.231534 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.231474 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5qjp\" (UniqueName: \"kubernetes.io/projected/a306921b-4523-4b47-915c-2b6251c276fc-kube-api-access-j5qjp\") pod \"payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf\" (UID: \"a306921b-4523-4b47-915c-2b6251c276fc\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" Nov 04 07:37:41.234442 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.233887 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" event={"ID":"5c14cd7f-ad7e-446c-be34-1b251628c970","Type":"ContainerStarted","Data":"f759f2df6e1d3800dbf6af939df5945a3b27ae6fa3d1b64a26790b0be33dfd22"} Nov 04 07:37:41.234442 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.233930 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" event={"ID":"5c14cd7f-ad7e-446c-be34-1b251628c970","Type":"ContainerStarted","Data":"c86d2da4a6647ae563cd65d19b97f7b8e6535bea26ab698f1271cad2b6181d1a"} Nov 04 07:37:41.234442 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.233986 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" Nov 04 07:37:41.259311 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.257910 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" podStartSLOduration=1.257890116 podStartE2EDuration="1.257890116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:41.252844795 +0000 UTC m=+1296.586784380" watchObservedRunningTime="2026-11-04 07:37:41.257890116 +0000 UTC m=+1296.591829701" Nov 04 07:37:41.333075 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.333041 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qstlq\" (UniqueName: \"kubernetes.io/projected/80160402-c66f-43db-8fd4-91fc30fa1460-kube-api-access-qstlq\") pod \"payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8\" (UID: \"80160402-c66f-43db-8fd4-91fc30fa1460\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" Nov 04 07:37:41.333265 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.333100 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5qjp\" (UniqueName: \"kubernetes.io/projected/a306921b-4523-4b47-915c-2b6251c276fc-kube-api-access-j5qjp\") pod \"payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf\" (UID: \"a306921b-4523-4b47-915c-2b6251c276fc\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" Nov 04 07:37:41.333345 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.333325 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a306921b-4523-4b47-915c-2b6251c276fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf\" (UID: \"a306921b-4523-4b47-915c-2b6251c276fc\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" Nov 04 07:37:41.333418 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.333404 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80160402-c66f-43db-8fd4-91fc30fa1460-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8\" (UID: \"80160402-c66f-43db-8fd4-91fc30fa1460\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" Nov 04 07:37:41.334301 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.334274 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80160402-c66f-43db-8fd4-91fc30fa1460-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8\" (UID: \"80160402-c66f-43db-8fd4-91fc30fa1460\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" Nov 04 07:37:41.334414 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.334274 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a306921b-4523-4b47-915c-2b6251c276fc-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf\" (UID: \"a306921b-4523-4b47-915c-2b6251c276fc\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" Nov 04 07:37:41.416531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.414254 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5qjp\" (UniqueName: \"kubernetes.io/projected/a306921b-4523-4b47-915c-2b6251c276fc-kube-api-access-j5qjp\") pod \"payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf\" (UID: \"a306921b-4523-4b47-915c-2b6251c276fc\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" Nov 04 07:37:41.416531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.414899 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qstlq\" (UniqueName: \"kubernetes.io/projected/80160402-c66f-43db-8fd4-91fc30fa1460-kube-api-access-qstlq\") pod \"payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8\" (UID: \"80160402-c66f-43db-8fd4-91fc30fa1460\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" Nov 04 07:37:41.478535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.477721 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" Nov 04 07:37:41.549705 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.549668 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" Nov 04 07:37:41.854687 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.854639 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8"] Nov 04 07:37:41.896729 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.896686 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:41.978334 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:41.943138 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf"] Nov 04 07:37:42.241670 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:42.241632 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" event={"ID":"a306921b-4523-4b47-915c-2b6251c276fc","Type":"ContainerStarted","Data":"a8796dbe3c2e536c17ebf4231971a8599b1e9c1eaea7ada593dc5042c087f92c"} Nov 04 07:37:42.241864 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:42.241730 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" event={"ID":"a306921b-4523-4b47-915c-2b6251c276fc","Type":"ContainerStarted","Data":"6f7e5af367e5c15c0c55454787e5de4a92bb27ca1887a788a755745c8a588bf2"} Nov 04 07:37:42.241864 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:42.241762 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" Nov 04 07:37:42.243116 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:42.243079 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" event={"ID":"80160402-c66f-43db-8fd4-91fc30fa1460","Type":"ContainerStarted","Data":"0f000461f6a6771380d8d4927d2af301215d3f224c80f67356dce413896b5222"} Nov 04 07:37:42.243226 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:42.243177 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" Nov 04 07:37:42.243226 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:42.243210 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" event={"ID":"80160402-c66f-43db-8fd4-91fc30fa1460","Type":"ContainerStarted","Data":"72e68c0b871d1e6ea93770386856e0d1d72a4ab098777f596d65cc2e451d6615"} Nov 04 07:37:42.295446 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:42.295367 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" podStartSLOduration=1.295317409 podStartE2EDuration="1.295317409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:42.273185449 +0000 UTC m=+1297.607125034" watchObservedRunningTime="2026-11-04 07:37:42.295317409 +0000 UTC m=+1297.629256994" Nov 04 07:37:42.300598 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:42.300521 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" podStartSLOduration=1.300489282 podStartE2EDuration="1.300489282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:42.298670513 +0000 UTC m=+1297.632610122" watchObservedRunningTime="2026-11-04 07:37:42.300489282 +0000 UTC m=+1297.634428868" Nov 04 07:37:42.965837 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:42.965803 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:43.694184 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:43.694149 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:44.390949 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.390910 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f"] Nov 04 07:37:44.396088 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.396064 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.407168 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.407139 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f"] Nov 04 07:37:44.508378 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.508342 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.508378 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.508380 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.508669 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.508478 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8cdda97-5400-4455-8741-9464bb3c430d-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.508669 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.508596 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.508787 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.508686 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.508787 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.508746 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.508898 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.508799 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8cdda97-5400-4455-8741-9464bb3c430d-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.508898 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.508885 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6kdm\" (UniqueName: \"kubernetes.io/projected/f8cdda97-5400-4455-8741-9464bb3c430d-kube-api-access-f6kdm\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.509000 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.508920 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8cdda97-5400-4455-8741-9464bb3c430d-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.520423 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.520395 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:44.610016 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.609969 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.610323 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.610284 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.610451 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.610354 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8cdda97-5400-4455-8741-9464bb3c430d-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.610451 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.610397 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.610685 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.610541 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.610685 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.610569 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.610685 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.610581 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.610864 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.610705 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8cdda97-5400-4455-8741-9464bb3c430d-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.610864 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.610780 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6kdm\" (UniqueName: \"kubernetes.io/projected/f8cdda97-5400-4455-8741-9464bb3c430d-kube-api-access-f6kdm\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.610864 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.610811 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8cdda97-5400-4455-8741-9464bb3c430d-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.610864 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.610855 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.611471 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.611447 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.611596 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.611492 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8cdda97-5400-4455-8741-9464bb3c430d-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.611993 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.611973 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.613173 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.613153 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8cdda97-5400-4455-8741-9464bb3c430d-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.613450 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.613432 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.623285 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.623263 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8cdda97-5400-4455-8741-9464bb3c430d-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.623490 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.623474 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6kdm\" (UniqueName: \"kubernetes.io/projected/f8cdda97-5400-4455-8741-9464bb3c430d-kube-api-access-f6kdm\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:44.714161 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:44.714098 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:45.002281 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:45.002251 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f"] Nov 04 07:37:45.003741 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:37:45.003715 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8cdda97_5400_4455_8741_9464bb3c430d.slice/crio-0be047a5a99d3930f5f7147cafcc83849a15308ca3c6db8614ed72e132cbee55 WatchSource:0}: Error finding container 0be047a5a99d3930f5f7147cafcc83849a15308ca3c6db8614ed72e132cbee55: Status 404 returned error can't find the container with id 0be047a5a99d3930f5f7147cafcc83849a15308ca3c6db8614ed72e132cbee55 Nov 04 07:37:45.006404 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:45.006374 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:37:45.006480 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:45.006439 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:37:45.006480 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:45.006469 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:37:45.262397 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:45.262297 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" event={"ID":"f8cdda97-5400-4455-8741-9464bb3c430d","Type":"ContainerStarted","Data":"570d95fd00ecd2a7ef2aea092e24a2439b6e394443bd4838422941d1071afd27"} Nov 04 07:37:45.262397 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:45.262389 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" event={"ID":"f8cdda97-5400-4455-8741-9464bb3c430d","Type":"ContainerStarted","Data":"0be047a5a99d3930f5f7147cafcc83849a15308ca3c6db8614ed72e132cbee55"} Nov 04 07:37:45.289780 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:45.289726 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" podStartSLOduration=1.289711886 podStartE2EDuration="1.289711886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:45.28688914 +0000 UTC m=+1300.620828739" watchObservedRunningTime="2026-11-04 07:37:45.289711886 +0000 UTC m=+1300.623651469" Nov 04 07:37:45.620466 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:45.620432 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:45.714273 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:45.714236 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:46.420927 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.420890 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc"] Nov 04 07:37:46.425171 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.425147 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.437039 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.437012 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc"] Nov 04 07:37:46.537658 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.537624 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.537842 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.537661 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99d41a69-904b-44b9-9a38-253d859dac34-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.537842 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.537686 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5dr5\" (UniqueName: \"kubernetes.io/projected/99d41a69-904b-44b9-9a38-253d859dac34-kube-api-access-d5dr5\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.537968 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.537851 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.537968 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.537897 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.538049 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.538024 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99d41a69-904b-44b9-9a38-253d859dac34-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.538088 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.538063 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99d41a69-904b-44b9-9a38-253d859dac34-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.538197 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.538182 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.538259 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.538210 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.604200 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.604161 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:46.639469 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.639439 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.639469 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.639470 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.639708 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.639549 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.639708 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.639577 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99d41a69-904b-44b9-9a38-253d859dac34-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.639708 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.639601 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5dr5\" (UniqueName: \"kubernetes.io/projected/99d41a69-904b-44b9-9a38-253d859dac34-kube-api-access-d5dr5\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.639708 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.639667 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.639907 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.639710 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.640036 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.639797 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99d41a69-904b-44b9-9a38-253d859dac34-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.640163 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.640062 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99d41a69-904b-44b9-9a38-253d859dac34-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.640163 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.640075 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.640163 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.640009 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.640357 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.640261 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.640420 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.640365 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.640669 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.640649 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99d41a69-904b-44b9-9a38-253d859dac34-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.641962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.641936 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.642494 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.642471 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99d41a69-904b-44b9-9a38-253d859dac34-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.686578 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.683651 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99d41a69-904b-44b9-9a38-253d859dac34-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.692916 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.692169 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5dr5\" (UniqueName: \"kubernetes.io/projected/99d41a69-904b-44b9-9a38-253d859dac34-kube-api-access-d5dr5\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:46.722484 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.722141 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Nov 04 07:37:46.722484 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.722221 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" podUID="f8cdda97-5400-4455-8741-9464bb3c430d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Nov 04 07:37:46.740791 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:46.740759 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:47.080538 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:47.078343 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc"] Nov 04 07:37:47.081464 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:37:47.081431 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99d41a69_904b_44b9_9a38_253d859dac34.slice/crio-63d8101c232663f8de1289334efa2ad2c9e68d23f8522b0f3f45e616cb5309b4 WatchSource:0}: Error finding container 63d8101c232663f8de1289334efa2ad2c9e68d23f8522b0f3f45e616cb5309b4: Status 404 returned error can't find the container with id 63d8101c232663f8de1289334efa2ad2c9e68d23f8522b0f3f45e616cb5309b4 Nov 04 07:37:47.084189 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:47.084151 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:37:47.084281 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:47.084238 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:37:47.084339 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:47.084278 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:37:47.291430 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:47.291385 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" event={"ID":"99d41a69-904b-44b9-9a38-253d859dac34","Type":"ContainerStarted","Data":"51e74e1e865eebec1c3095c6162bccb6191ffa5b84db749f4d481f4d0fbe1f9c"} Nov 04 07:37:47.291638 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:47.291449 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" event={"ID":"99d41a69-904b-44b9-9a38-253d859dac34","Type":"ContainerStarted","Data":"63d8101c232663f8de1289334efa2ad2c9e68d23f8522b0f3f45e616cb5309b4"} Nov 04 07:37:47.597745 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:47.597699 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:47.699001 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:47.698972 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:47.715270 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:47.715237 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Nov 04 07:37:47.715432 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:47.715297 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" podUID="f8cdda97-5400-4455-8741-9464bb3c430d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Nov 04 07:37:47.742786 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:47.742753 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:47.805528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:47.805082 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:48.608528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.605750 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" podStartSLOduration=2.605731175 podStartE2EDuration="2.605731175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:47.360188563 +0000 UTC m=+1302.694128149" watchObservedRunningTime="2026-11-04 07:37:48.605731175 +0000 UTC m=+1303.939670759" Nov 04 07:37:48.610563 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.609923 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb"] Nov 04 07:37:48.616041 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.616015 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:37:48.628524 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.627578 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-3da894-serving-cert\"" Nov 04 07:37:48.630299 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.630267 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb"] Nov 04 07:37:48.698564 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.698389 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:48.730562 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.724935 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Nov 04 07:37:48.730562 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.725009 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" podUID="f8cdda97-5400-4455-8741-9464bb3c430d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Nov 04 07:37:48.772952 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.772916 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Nov 04 07:37:48.773620 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.773581 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" podUID="99d41a69-904b-44b9-9a38-253d859dac34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Nov 04 07:37:48.823735 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.801470 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-maas-api-tls\") pod \"maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb\" (UID: \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:37:48.823735 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.804175 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbc89\" (UniqueName: \"kubernetes.io/projected/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-kube-api-access-dbc89\") pod \"maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb\" (UID: \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:37:48.823735 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.804254 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-metrics-certs\") pod \"maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb\" (UID: \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:37:48.823735 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.804367 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb\" (UID: \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:37:48.823735 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.810352 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:48.906314 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.906272 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-metrics-certs\") pod \"maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb\" (UID: \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:37:48.907228 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.907200 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb\" (UID: \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:37:48.908288 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.908267 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-maas-api-tls\") pod \"maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb\" (UID: \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:37:48.943659 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.943628 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbc89\" (UniqueName: \"kubernetes.io/projected/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-kube-api-access-dbc89\") pod \"maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb\" (UID: \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:37:48.943978 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.926196 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:48.944142 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.908094 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb\" (UID: \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:37:48.944250 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.943398 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-metrics-certs\") pod \"maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb\" (UID: \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:37:48.945170 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.945036 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-maas-api-tls\") pod \"maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb\" (UID: \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:37:49.010532 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:48.982187 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbc89\" (UniqueName: \"kubernetes.io/projected/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-kube-api-access-dbc89\") pod \"maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb\" (UID: \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:37:49.253863 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:49.253762 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:37:49.647531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:49.642253 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb"] Nov 04 07:37:49.732206 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:49.731538 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Nov 04 07:37:49.732206 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:49.731589 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" podUID="f8cdda97-5400-4455-8741-9464bb3c430d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Nov 04 07:37:49.741107 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:49.741081 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Nov 04 07:37:49.741235 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:49.741128 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" podUID="99d41a69-904b-44b9-9a38-253d859dac34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Nov 04 07:37:50.259574 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:50.254115 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:50.383528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:50.380252 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" event={"ID":"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909","Type":"ContainerStarted","Data":"1964b60a55904ebf47ceb40ea0c73d493495455168f48a806aabaa048cb17d83"} Nov 04 07:37:50.716049 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:50.715592 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Nov 04 07:37:50.716049 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:50.715657 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" podUID="f8cdda97-5400-4455-8741-9464bb3c430d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Nov 04 07:37:50.741916 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:50.741875 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Nov 04 07:37:50.742067 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:50.741940 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" podUID="99d41a69-904b-44b9-9a38-253d859dac34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Nov 04 07:37:51.416530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:51.416030 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" event={"ID":"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909","Type":"ContainerStarted","Data":"21e999625e0ae2335a80f4aa981cd264673b8ecdb434e60b54a1acf2019cb5ee"} Nov 04 07:37:51.418544 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:51.417081 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:37:51.440409 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:51.440359 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" podStartSLOduration=2.83796473 podStartE2EDuration="3.44034516s" totalImagesPullingTime="602.38043ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:37:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:37:51.437442549 +0000 UTC m=+1306.771382124" watchObservedRunningTime="2026-11-04 07:37:51.44034516 +0000 UTC m=+1306.774284744" Nov 04 07:37:51.525767 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:51.525728 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:51.715352 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:51.715256 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Nov 04 07:37:51.715352 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:51.715326 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" podUID="f8cdda97-5400-4455-8741-9464bb3c430d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Nov 04 07:37:51.742104 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:51.742057 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Nov 04 07:37:51.742269 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:51.742126 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" podUID="99d41a69-904b-44b9-9a38-253d859dac34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Nov 04 07:37:52.288531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:52.275413 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" Nov 04 07:37:52.288531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:52.280697 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:37:52.726606 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:52.716903 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Nov 04 07:37:52.726606 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:52.716983 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" podUID="f8cdda97-5400-4455-8741-9464bb3c430d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Nov 04 07:37:52.744159 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:52.743834 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Nov 04 07:37:52.744159 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:52.743893 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" podUID="99d41a69-904b-44b9-9a38-253d859dac34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Nov 04 07:37:53.129545 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:53.122212 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:53.277629 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:53.277583 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" Nov 04 07:37:53.284180 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:53.284105 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" Nov 04 07:37:53.781057 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:53.781019 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Nov 04 07:37:53.781675 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:53.781642 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" podUID="99d41a69-904b-44b9-9a38-253d859dac34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Nov 04 07:37:54.059912 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:54.059824 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:54.066817 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:54.066787 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:54.071762 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:54.071734 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:37:54.467225 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:54.467187 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:54.742027 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:54.741941 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Nov 04 07:37:54.742172 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:54.742020 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" podUID="99d41a69-904b-44b9-9a38-253d859dac34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Nov 04 07:37:55.315077 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:55.315047 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:55.912568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:55.912533 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:55.912997 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:55.912970 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:55.914080 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:55.914057 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:37:56.384732 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:56.384698 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:57.178555 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:57.178500 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:57.883791 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:57.883758 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:57.987978 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:57.987947 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:58.693751 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:58.693723 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:37:58.798977 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:58.798936 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:59.135559 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:59.135521 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:59.260307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:59.260072 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:59.429247 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:59.429163 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:59.739107 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:59.739004 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:37:59.918168 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:37:59.918123 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:00.121585 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:00.120107 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:00.243296 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:00.243260 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:00.465757 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:00.465662 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:00.573489 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:00.573458 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:00.880566 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:38:00.880523 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:38:00.931135 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:00.931106 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:01.068434 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.068391 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz"] Nov 04 07:38:01.073544 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.073497 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.090913 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.090883 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz"] Nov 04 07:38:01.268881 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.268787 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.269473 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.269447 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55jdb\" (UniqueName: \"kubernetes.io/projected/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-kube-api-access-55jdb\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.269691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.269671 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.269827 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.269808 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.269967 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.269951 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.270157 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.270141 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.270265 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.270252 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.270399 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.270386 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.270535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.270519 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.373696 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.373148 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.376110 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.376079 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.377319 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.377294 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.378060 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.378037 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.378558 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.378539 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.378715 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.378695 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55jdb\" (UniqueName: \"kubernetes.io/projected/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-kube-api-access-55jdb\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.378886 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.378868 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.380027 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.380008 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.380649 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.380568 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.381307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.378423 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-credential-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.381469 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.379945 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-workload-socket\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.381625 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.375080 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.381878 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.381119 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-istio-data\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.382009 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.377953 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-workload-certs\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.409641 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.409561 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-istio-podinfo\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.451124 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.451079 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-istio-token\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.453548 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.453521 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55jdb\" (UniqueName: \"kubernetes.io/projected/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-kube-api-access-55jdb\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.488584 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.467808 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e734faeb-28a7-4aa2-86cc-dc9c24cf5c34-istio-envoy\") pod \"maas-default-gateway-openshift-default-74854d8ccf-bx8vz\" (UID: \"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:01.612749 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.612667 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:01.693884 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:01.693847 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:02.081835 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:02.081798 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz"] Nov 04 07:38:02.083921 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:38:02.083896 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode734faeb_28a7_4aa2_86cc_dc9c24cf5c34.slice/crio-6899d9bba31def587fe0c108f777221bd319de57f65786c0831690594e532101 WatchSource:0}: Error finding container 6899d9bba31def587fe0c108f777221bd319de57f65786c0831690594e532101: Status 404 returned error can't find the container with id 6899d9bba31def587fe0c108f777221bd319de57f65786c0831690594e532101 Nov 04 07:38:02.086779 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:02.086742 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:38:02.086867 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:02.086827 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:38:02.086910 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:02.086870 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:38:02.685736 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:02.685702 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:02.803527 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:02.798182 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" event={"ID":"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34","Type":"ContainerStarted","Data":"03425f273e3364be6b3f06506ad07d13b6ac70cb4b8f2f82816a7509f885fe65"} Nov 04 07:38:02.803527 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:02.798302 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" event={"ID":"e734faeb-28a7-4aa2-86cc-dc9c24cf5c34","Type":"ContainerStarted","Data":"6899d9bba31def587fe0c108f777221bd319de57f65786c0831690594e532101"} Nov 04 07:38:03.696962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:03.696924 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:03.698539 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:03.698494 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bx8vz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Nov 04 07:38:03.698661 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:03.698559 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" podUID="e734faeb-28a7-4aa2-86cc-dc9c24cf5c34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Nov 04 07:38:03.743881 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:03.743848 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:04.293926 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:04.293890 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:04.442610 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:04.442576 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:04.711556 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:04.709642 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bx8vz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Nov 04 07:38:04.711556 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:04.709706 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" podUID="e734faeb-28a7-4aa2-86cc-dc9c24cf5c34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Nov 04 07:38:04.841983 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:04.841945 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:05.527184 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:05.527154 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:05.639779 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:05.639753 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:05.715672 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:05.715635 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bx8vz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Nov 04 07:38:05.716271 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:05.716236 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" podUID="e734faeb-28a7-4aa2-86cc-dc9c24cf5c34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Nov 04 07:38:06.016527 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:05.994579 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" podStartSLOduration=4.994558451 podStartE2EDuration="4.994558451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:02.841657706 +0000 UTC m=+1318.175597290" watchObservedRunningTime="2026-11-04 07:38:05.994558451 +0000 UTC m=+1321.328498036" Nov 04 07:38:06.028617 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.028566 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b"] Nov 04 07:38:06.043722 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.043691 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b"] Nov 04 07:38:06.044041 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.044022 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.132190 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.132153 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-be2a74-openshift-default-dockercfg-npbmk\"" Nov 04 07:38:06.135806 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.135776 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-token\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.135998 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.135982 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-podinfo\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.136157 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.136141 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-envoy\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.136302 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.136286 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-workload-socket\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.136399 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.136385 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-workload-certs\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.136604 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.136575 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-data\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.136735 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.136720 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8a5b7b3-360d-475c-87b7-6a845394a09c-istiod-ca-cert\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.136862 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.136847 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgkkp\" (UniqueName: \"kubernetes.io/projected/e8a5b7b3-360d-475c-87b7-6a845394a09c-kube-api-access-kgkkp\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.137004 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.136989 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-credential-socket\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.143672 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.143647 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:06.239270 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.238266 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-workload-socket\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.239270 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.238314 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-workload-certs\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.239270 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.238403 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-data\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.239270 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.238430 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8a5b7b3-360d-475c-87b7-6a845394a09c-istiod-ca-cert\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.239270 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.238468 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgkkp\" (UniqueName: \"kubernetes.io/projected/e8a5b7b3-360d-475c-87b7-6a845394a09c-kube-api-access-kgkkp\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.239270 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.238551 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-credential-socket\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.239270 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.238655 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-token\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.239270 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.238684 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-podinfo\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.242338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.238771 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-envoy\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.242338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.240959 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8a5b7b3-360d-475c-87b7-6a845394a09c-istiod-ca-cert\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.242338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.241223 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-workload-socket\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.242338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.241480 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-workload-certs\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.244074 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.242942 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-data\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.244074 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.243206 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-credential-socket\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.252215 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.252048 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-envoy\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.275449 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.275369 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:06.291535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.291483 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-token\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.312709 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.292490 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-podinfo\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.325554 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.313640 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgkkp\" (UniqueName: \"kubernetes.io/projected/e8a5b7b3-360d-475c-87b7-6a845394a09c-kube-api-access-kgkkp\") pod \"e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.542964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.542861 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:06.697806 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.697760 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bx8vz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Nov 04 07:38:06.697991 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.697854 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" podUID="e734faeb-28a7-4aa2-86cc-dc9c24cf5c34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Nov 04 07:38:06.762304 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:06.762226 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:07.008258 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:07.008116 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:07.011041 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:07.010961 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b"] Nov 04 07:38:07.087534 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:38:07.079852 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8a5b7b3_360d_475c_87b7_6a845394a09c.slice/crio-c1c60ff54ada6bf9df85388beefe3d64de71009b949a312e5be62f7d95f8b03c WatchSource:0}: Error finding container c1c60ff54ada6bf9df85388beefe3d64de71009b949a312e5be62f7d95f8b03c: Status 404 returned error can't find the container with id c1c60ff54ada6bf9df85388beefe3d64de71009b949a312e5be62f7d95f8b03c Nov 04 07:38:07.087534 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:07.085928 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:38:07.087534 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:07.086014 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:38:07.087534 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:07.086057 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:38:07.126578 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:07.126549 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:07.697722 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:07.697652 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bx8vz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Nov 04 07:38:07.697926 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:07.697728 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" podUID="e734faeb-28a7-4aa2-86cc-dc9c24cf5c34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Nov 04 07:38:07.918153 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:07.918118 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:08.091146 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:08.091053 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" event={"ID":"e8a5b7b3-360d-475c-87b7-6a845394a09c","Type":"ContainerStarted","Data":"4d1b26e1e87ad26f5be3d994366c5f8a1ce778ffc5dbe2df5a0268842ef4c4f8"} Nov 04 07:38:08.091146 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:08.091121 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" event={"ID":"e8a5b7b3-360d-475c-87b7-6a845394a09c","Type":"ContainerStarted","Data":"c1c60ff54ada6bf9df85388beefe3d64de71009b949a312e5be62f7d95f8b03c"} Nov 04 07:38:08.119906 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:08.119851 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" podStartSLOduration=3.119836712 podStartE2EDuration="3.119836712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:08.118321732 +0000 UTC m=+1323.452261341" watchObservedRunningTime="2026-11-04 07:38:08.119836712 +0000 UTC m=+1323.453776296" Nov 04 07:38:08.543890 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:08.543848 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:08.549364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:08.549338 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:08.697468 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:08.697434 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bx8vz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Nov 04 07:38:08.697647 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:08.697486 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" podUID="e734faeb-28a7-4aa2-86cc-dc9c24cf5c34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Nov 04 07:38:08.874306 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:08.874276 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:09.098138 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:09.098097 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:09.098917 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:09.098898 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:09.697304 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:09.697271 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bx8vz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body= Nov 04 07:38:09.697477 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:09.697324 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" podUID="e734faeb-28a7-4aa2-86cc-dc9c24cf5c34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" Nov 04 07:38:09.988823 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:09.988739 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:10.475055 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:10.475005 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:10.584284 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:10.584245 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:11.086488 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:11.086455 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:11.698296 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:11.698251 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bx8vz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 07:38:11.698827 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:11.698313 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" podUID="e734faeb-28a7-4aa2-86cc-dc9c24cf5c34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 07:38:12.181733 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:12.181694 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:12.740704 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:12.740663 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bx8vz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 07:38:12.741213 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:12.740740 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" podUID="e734faeb-28a7-4aa2-86cc-dc9c24cf5c34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 07:38:13.158481 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:13.158435 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:13.446880 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:13.446759 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:13.581111 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:13.581012 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:13.719594 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:13.719425 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:13.797529 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:13.787637 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-bx8vz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 07:38:13.797529 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:13.787706 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" podUID="e734faeb-28a7-4aa2-86cc-dc9c24cf5c34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 07:38:14.155631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:14.155603 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:14.156113 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:14.156090 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:14.156833 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:14.156811 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-bx8vz" Nov 04 07:38:15.087954 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:15.087919 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:15.403249 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:15.403218 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:15.474657 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:15.474625 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5"] Nov 04 07:38:15.479573 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:15.479544 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" Nov 04 07:38:15.486984 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:15.486957 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-023307-dockercfg-jsbwr\"" Nov 04 07:38:15.488825 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:15.488800 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5"] Nov 04 07:38:15.490548 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:15.490529 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-023307\"" Nov 04 07:38:15.533432 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:15.533405 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:15.558018 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:15.557991 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5hfd\" (UniqueName: \"kubernetes.io/projected/60ad1363-dd96-42e2-89ae-7c50adfae243-kube-api-access-s5hfd\") pod \"payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5\" (UID: \"60ad1363-dd96-42e2-89ae-7c50adfae243\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" Nov 04 07:38:15.558175 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:15.558149 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60ad1363-dd96-42e2-89ae-7c50adfae243-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5\" (UID: \"60ad1363-dd96-42e2-89ae-7c50adfae243\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" Nov 04 07:38:15.659405 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:15.659324 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5hfd\" (UniqueName: \"kubernetes.io/projected/60ad1363-dd96-42e2-89ae-7c50adfae243-kube-api-access-s5hfd\") pod \"payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5\" (UID: \"60ad1363-dd96-42e2-89ae-7c50adfae243\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" Nov 04 07:38:15.659571 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:15.659424 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60ad1363-dd96-42e2-89ae-7c50adfae243-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5\" (UID: \"60ad1363-dd96-42e2-89ae-7c50adfae243\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" Nov 04 07:38:15.660217 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:15.660192 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60ad1363-dd96-42e2-89ae-7c50adfae243-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5\" (UID: \"60ad1363-dd96-42e2-89ae-7c50adfae243\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" Nov 04 07:38:15.673552 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:15.673524 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5hfd\" (UniqueName: \"kubernetes.io/projected/60ad1363-dd96-42e2-89ae-7c50adfae243-kube-api-access-s5hfd\") pod \"payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5\" (UID: \"60ad1363-dd96-42e2-89ae-7c50adfae243\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" Nov 04 07:38:15.796652 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:15.796619 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" Nov 04 07:38:16.121815 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:16.121782 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5"] Nov 04 07:38:16.124539 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:38:16.124498 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60ad1363_dd96_42e2_89ae_7c50adfae243.slice/crio-5a75fbdb11e8ca7c30c137e930d83243aa1dc481eccdf8f5f63ad5380a7f2006 WatchSource:0}: Error finding container 5a75fbdb11e8ca7c30c137e930d83243aa1dc481eccdf8f5f63ad5380a7f2006: Status 404 returned error can't find the container with id 5a75fbdb11e8ca7c30c137e930d83243aa1dc481eccdf8f5f63ad5380a7f2006 Nov 04 07:38:16.199493 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:16.199464 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" event={"ID":"60ad1363-dd96-42e2-89ae-7c50adfae243","Type":"ContainerStarted","Data":"5a75fbdb11e8ca7c30c137e930d83243aa1dc481eccdf8f5f63ad5380a7f2006"} Nov 04 07:38:16.457580 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:16.456752 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:16.557331 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:16.557299 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:17.207614 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:17.207578 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" event={"ID":"60ad1363-dd96-42e2-89ae-7c50adfae243","Type":"ContainerStarted","Data":"76e9558329c5d0a2b71c4d3246830945b423f91e3b4fd9d04794242857138e33"} Nov 04 07:38:17.208024 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:17.207653 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" Nov 04 07:38:17.228477 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:17.228419 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" podStartSLOduration=2.228406768 podStartE2EDuration="2.228406768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:17.226693748 +0000 UTC m=+1332.560633331" watchObservedRunningTime="2026-11-04 07:38:17.228406768 +0000 UTC m=+1332.562346352" Nov 04 07:38:17.398717 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:17.398683 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:18.284691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:18.284655 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:19.087457 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:19.087402 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:19.980324 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:19.980291 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:20.806515 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:20.806466 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t"] Nov 04 07:38:20.812221 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:20.812171 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:38:20.823403 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:20.823375 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-be2a74-serving-cert\"" Nov 04 07:38:20.841597 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:20.841568 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t"] Nov 04 07:38:20.864551 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:20.864497 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:20.950877 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:20.950844 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-metrics-certs\") pod \"maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t\" (UID: \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:38:20.951086 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:20.951057 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z45bf\" (UniqueName: \"kubernetes.io/projected/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-kube-api-access-z45bf\") pod \"maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t\" (UID: \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:38:20.951358 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:20.951331 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t\" (UID: \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:38:20.951546 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:20.951524 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-maas-api-tls\") pod \"maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t\" (UID: \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:38:20.997091 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:20.997013 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:21.054726 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:21.054685 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-maas-api-tls\") pod \"maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t\" (UID: \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:38:21.054925 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:21.054804 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-metrics-certs\") pod \"maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t\" (UID: \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:38:21.054994 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:21.054927 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z45bf\" (UniqueName: \"kubernetes.io/projected/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-kube-api-access-z45bf\") pod \"maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t\" (UID: \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:38:21.055047 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:21.054994 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t\" (UID: \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:38:21.055871 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:21.055843 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t\" (UID: \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:38:21.066003 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:21.065916 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-maas-api-tls\") pod \"maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t\" (UID: \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:38:21.069791 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:21.069764 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-metrics-certs\") pod \"maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t\" (UID: \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:38:21.074608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:21.074533 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z45bf\" (UniqueName: \"kubernetes.io/projected/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-kube-api-access-z45bf\") pod \"maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t\" (UID: \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:38:21.142854 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:21.142824 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:38:21.451190 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:21.447836 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t"] Nov 04 07:38:21.788896 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:21.788799 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:22.275792 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:22.275759 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" event={"ID":"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0","Type":"ContainerStarted","Data":"f8e0458a074079f228a7c4bf649b1da03b827faa22fe6eace4896997c942c02f"} Nov 04 07:38:22.276168 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:22.275810 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" event={"ID":"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0","Type":"ContainerStarted","Data":"022beb131787804b0fd1665d6d1cd74b7ca0cbb9e556307250ea5d9152d2ac64"} Nov 04 07:38:22.276168 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:22.275843 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:38:22.298006 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:22.297960 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" podStartSLOduration=1.7086485439999999 podStartE2EDuration="2.297946971s" totalImagesPullingTime="589.298427ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:22.296253516 +0000 UTC m=+1337.630193125" watchObservedRunningTime="2026-11-04 07:38:22.297946971 +0000 UTC m=+1337.631886556" Nov 04 07:38:22.688888 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:22.688857 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:23.517970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:23.513536 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:24.325089 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.325054 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv"] Nov 04 07:38:24.329129 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.329103 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:38:24.342713 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.342686 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-023307-serving-cert\"" Nov 04 07:38:24.347876 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.347847 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv"] Nov 04 07:38:24.382689 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.382657 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:24.431680 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.431645 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cntrt\" (UniqueName: \"kubernetes.io/projected/481004b6-384d-45d9-b7b5-8f79abfd1c06-kube-api-access-cntrt\") pod \"maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv\" (UID: \"481004b6-384d-45d9-b7b5-8f79abfd1c06\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:38:24.431680 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.431677 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/481004b6-384d-45d9-b7b5-8f79abfd1c06-maas-api-tls\") pod \"maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv\" (UID: \"481004b6-384d-45d9-b7b5-8f79abfd1c06\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:38:24.431897 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.431804 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/481004b6-384d-45d9-b7b5-8f79abfd1c06-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv\" (UID: \"481004b6-384d-45d9-b7b5-8f79abfd1c06\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:38:24.431939 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.431926 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/481004b6-384d-45d9-b7b5-8f79abfd1c06-metrics-certs\") pod \"maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv\" (UID: \"481004b6-384d-45d9-b7b5-8f79abfd1c06\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:38:24.526816 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.524088 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:24.534456 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.534414 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/481004b6-384d-45d9-b7b5-8f79abfd1c06-metrics-certs\") pod \"maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv\" (UID: \"481004b6-384d-45d9-b7b5-8f79abfd1c06\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:38:24.534704 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.534639 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cntrt\" (UniqueName: \"kubernetes.io/projected/481004b6-384d-45d9-b7b5-8f79abfd1c06-kube-api-access-cntrt\") pod \"maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv\" (UID: \"481004b6-384d-45d9-b7b5-8f79abfd1c06\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:38:24.534704 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.534672 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/481004b6-384d-45d9-b7b5-8f79abfd1c06-maas-api-tls\") pod \"maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv\" (UID: \"481004b6-384d-45d9-b7b5-8f79abfd1c06\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:38:24.534832 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.534748 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/481004b6-384d-45d9-b7b5-8f79abfd1c06-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv\" (UID: \"481004b6-384d-45d9-b7b5-8f79abfd1c06\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:38:24.536687 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.535716 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/481004b6-384d-45d9-b7b5-8f79abfd1c06-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv\" (UID: \"481004b6-384d-45d9-b7b5-8f79abfd1c06\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:38:24.539852 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.539733 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/481004b6-384d-45d9-b7b5-8f79abfd1c06-metrics-certs\") pod \"maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv\" (UID: \"481004b6-384d-45d9-b7b5-8f79abfd1c06\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:38:24.540556 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.540481 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/481004b6-384d-45d9-b7b5-8f79abfd1c06-maas-api-tls\") pod \"maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv\" (UID: \"481004b6-384d-45d9-b7b5-8f79abfd1c06\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:38:24.550423 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.550399 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cntrt\" (UniqueName: \"kubernetes.io/projected/481004b6-384d-45d9-b7b5-8f79abfd1c06-kube-api-access-cntrt\") pod \"maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv\" (UID: \"481004b6-384d-45d9-b7b5-8f79abfd1c06\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:38:24.635178 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.635144 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:24.646735 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:24.646707 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:38:25.041535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:25.034754 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv"] Nov 04 07:38:25.295935 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:25.295845 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" event={"ID":"481004b6-384d-45d9-b7b5-8f79abfd1c06","Type":"ContainerStarted","Data":"46dcbe28c25aed98a4e1ddb1b2590583823ca75e5ef257770478f8371f2927f7"} Nov 04 07:38:25.399882 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:25.399852 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:26.303777 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:26.303742 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" event={"ID":"481004b6-384d-45d9-b7b5-8f79abfd1c06","Type":"ContainerStarted","Data":"b2e55daed19627a363df80c7f4ace05c77691dd3f764e3185fc3d712d64a2855"} Nov 04 07:38:26.304189 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:26.303894 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:38:26.326102 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:26.326055 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" podStartSLOduration=1.88141543 podStartE2EDuration="2.326040614s" totalImagesPullingTime="444.625184ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:26.323237876 +0000 UTC m=+1341.657177459" watchObservedRunningTime="2026-11-04 07:38:26.326040614 +0000 UTC m=+1341.659980199" Nov 04 07:38:26.666418 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:26.666383 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:27.565838 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:27.565805 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:28.217764 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:28.217737 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" Nov 04 07:38:28.287079 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:28.287050 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:38:28.341260 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:28.341223 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:29.165881 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:29.165850 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:29.977089 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:29.977052 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:30.764759 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:30.764725 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:30.906495 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:38:30.906460 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:38:30.913213 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:30.913183 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:31.553882 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:31.553849 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:31.756869 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:31.756836 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:31.921794 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:31.921764 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:32.316715 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:32.316647 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:38:32.857080 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:32.857040 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:33.787851 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:33.787812 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:34.679048 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:34.679012 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:35.574804 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:35.574773 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:35.682914 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:35.682877 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:36.069325 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:36.069291 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:36.177768 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:36.177738 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:36.558316 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:36.558278 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:36.793666 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:36.793633 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:36.905853 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:36.905824 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:37.092253 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:37.092219 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:37.458868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:37.458835 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:37.670947 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:37.670912 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:37.799868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:37.799785 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:38.418471 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:38.418436 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:38.552911 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:38.552876 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:38.850837 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:38.850747 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:39.124942 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:39.124887 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:39.742531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:39.735096 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:40.084581 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:40.084224 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:40.246951 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:40.246921 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:40.394752 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:40.394722 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:40.863801 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:40.863724 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:41.026292 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:41.026260 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:41.460106 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:41.460076 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:41.581499 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:41.581463 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:42.182064 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.182028 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww"] Nov 04 07:38:42.187560 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.187530 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:38:42.197061 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.196971 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-023307\"/\"agc-cred-eb4d01a4e8eb\"" Nov 04 07:38:42.197973 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.197942 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-023307\"/\"payload-processing-external-model-e2e--80f769cf-dockercfg-9gmj8\"" Nov 04 07:38:42.199804 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.199776 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-023307\"/\"kube-root-ca.crt\"" Nov 04 07:38:42.199948 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.199816 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-023307\"/\"payload-processing-external-model-plugins-e2e-extmod-b-023307\"" Nov 04 07:38:42.199948 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.199888 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-023307\"/\"openshift-service-ca.crt\"" Nov 04 07:38:42.199948 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.199898 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-023307\"/\"routing-overlay\"" Nov 04 07:38:42.201680 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.201659 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww"] Nov 04 07:38:42.274754 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.274715 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/808ce2ba-76f7-4825-986d-4d1030e4737e-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww\" (UID: \"808ce2ba-76f7-4825-986d-4d1030e4737e\") " pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:38:42.274942 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.274877 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/808ce2ba-76f7-4825-986d-4d1030e4737e-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww\" (UID: \"808ce2ba-76f7-4825-986d-4d1030e4737e\") " pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:38:42.275022 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.275000 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsbbm\" (UniqueName: \"kubernetes.io/projected/808ce2ba-76f7-4825-986d-4d1030e4737e-kube-api-access-hsbbm\") pod \"payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww\" (UID: \"808ce2ba-76f7-4825-986d-4d1030e4737e\") " pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:38:42.275088 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.275066 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/808ce2ba-76f7-4825-986d-4d1030e4737e-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww\" (UID: \"808ce2ba-76f7-4825-986d-4d1030e4737e\") " pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:38:42.380548 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.380480 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/808ce2ba-76f7-4825-986d-4d1030e4737e-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww\" (UID: \"808ce2ba-76f7-4825-986d-4d1030e4737e\") " pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:38:42.380889 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.380867 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsbbm\" (UniqueName: \"kubernetes.io/projected/808ce2ba-76f7-4825-986d-4d1030e4737e-kube-api-access-hsbbm\") pod \"payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww\" (UID: \"808ce2ba-76f7-4825-986d-4d1030e4737e\") " pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:38:42.381025 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.381009 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/808ce2ba-76f7-4825-986d-4d1030e4737e-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww\" (UID: \"808ce2ba-76f7-4825-986d-4d1030e4737e\") " pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:38:42.381244 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.381225 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/808ce2ba-76f7-4825-986d-4d1030e4737e-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww\" (UID: \"808ce2ba-76f7-4825-986d-4d1030e4737e\") " pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:38:42.384333 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.382884 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/808ce2ba-76f7-4825-986d-4d1030e4737e-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww\" (UID: \"808ce2ba-76f7-4825-986d-4d1030e4737e\") " pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:38:42.384333 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.383547 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/808ce2ba-76f7-4825-986d-4d1030e4737e-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww\" (UID: \"808ce2ba-76f7-4825-986d-4d1030e4737e\") " pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:38:42.387206 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.387178 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/808ce2ba-76f7-4825-986d-4d1030e4737e-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww\" (UID: \"808ce2ba-76f7-4825-986d-4d1030e4737e\") " pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:38:42.422138 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.422097 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsbbm\" (UniqueName: \"kubernetes.io/projected/808ce2ba-76f7-4825-986d-4d1030e4737e-kube-api-access-hsbbm\") pod \"payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww\" (UID: \"808ce2ba-76f7-4825-986d-4d1030e4737e\") " pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:38:42.504028 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.503934 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:38:42.822488 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:42.822455 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww"] Nov 04 07:38:42.823473 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:38:42.823448 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod808ce2ba_76f7_4825_986d_4d1030e4737e.slice/crio-0a6ffa64c9b026c88cbe3ab9a680b77507bc16aa5735a41b7c22d31aa171c936 WatchSource:0}: Error finding container 0a6ffa64c9b026c88cbe3ab9a680b77507bc16aa5735a41b7c22d31aa171c936: Status 404 returned error can't find the container with id 0a6ffa64c9b026c88cbe3ab9a680b77507bc16aa5735a41b7c22d31aa171c936 Nov 04 07:38:43.432950 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:43.432920 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:43.479586 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:43.479541 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" event={"ID":"808ce2ba-76f7-4825-986d-4d1030e4737e","Type":"ContainerStarted","Data":"e08df657e4ceab34f1c5cb5d884d86efb92a4865aab2166e5aaf48a6230dca5f"} Nov 04 07:38:43.479797 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:43.479615 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" event={"ID":"808ce2ba-76f7-4825-986d-4d1030e4737e","Type":"ContainerStarted","Data":"0a6ffa64c9b026c88cbe3ab9a680b77507bc16aa5735a41b7c22d31aa171c936"} Nov 04 07:38:43.479797 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:43.479755 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:38:43.513588 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:43.513529 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" podStartSLOduration=1.513515065 podStartE2EDuration="1.513515065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:43.510940425 +0000 UTC m=+1358.844880030" watchObservedRunningTime="2026-11-04 07:38:43.513515065 +0000 UTC m=+1358.847454639" Nov 04 07:38:43.571614 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:43.571571 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:43.884214 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:43.884160 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd"] Nov 04 07:38:43.918926 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:43.918889 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:44.016673 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:44.016631 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:44.696665 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:44.696633 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:45.611375 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:45.611346 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:45.715345 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:45.715311 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:46.581556 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:46.581453 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:47.568530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:47.561977 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:48.497985 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:48.497949 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:48.884914 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:48.884876 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" podUID="87e99792-812e-4038-8262-13eaf2a52347" containerName="payload-processing" containerID="cri-o://46b14ffbe2fd6231f38019d565febbdeb6c0c380b3c6bce10525f4b4265f102a" gracePeriod=25 Nov 04 07:38:49.175740 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.175710 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:38:49.246448 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.246410 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87e99792-812e-4038-8262-13eaf2a52347-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87e99792-812e-4038-8262-13eaf2a52347-plugins-config-volume\") pod \"87e99792-812e-4038-8262-13eaf2a52347\" (UID: \"87e99792-812e-4038-8262-13eaf2a52347\") " Nov 04 07:38:49.246660 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.246461 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/87e99792-812e-4038-8262-13eaf2a52347-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/87e99792-812e-4038-8262-13eaf2a52347-routing-overlay-volume\") pod \"87e99792-812e-4038-8262-13eaf2a52347\" (UID: \"87e99792-812e-4038-8262-13eaf2a52347\") " Nov 04 07:38:49.246660 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.246556 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87e99792-812e-4038-8262-13eaf2a52347-provider-credentials\" (UniqueName: \"kubernetes.io/projected/87e99792-812e-4038-8262-13eaf2a52347-provider-credentials\") pod \"87e99792-812e-4038-8262-13eaf2a52347\" (UID: \"87e99792-812e-4038-8262-13eaf2a52347\") " Nov 04 07:38:49.246660 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.246618 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/87e99792-812e-4038-8262-13eaf2a52347-kube-api-access-bdbpf\" (UniqueName: \"kubernetes.io/projected/87e99792-812e-4038-8262-13eaf2a52347-kube-api-access-bdbpf\") pod \"87e99792-812e-4038-8262-13eaf2a52347\" (UID: \"87e99792-812e-4038-8262-13eaf2a52347\") " Nov 04 07:38:49.246953 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.246924 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87e99792-812e-4038-8262-13eaf2a52347-routing-overlay-volume" pod "87e99792-812e-4038-8262-13eaf2a52347" (UID: "87e99792-812e-4038-8262-13eaf2a52347"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:38:49.247049 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.246935 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/87e99792-812e-4038-8262-13eaf2a52347-plugins-config-volume" pod "87e99792-812e-4038-8262-13eaf2a52347" (UID: "87e99792-812e-4038-8262-13eaf2a52347"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:38:49.247196 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.247153 5385 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/87e99792-812e-4038-8262-13eaf2a52347-plugins-config-volume\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:38:49.247196 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.247171 5385 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/87e99792-812e-4038-8262-13eaf2a52347-routing-overlay-volume\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:38:49.249462 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.249438 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87e99792-812e-4038-8262-13eaf2a52347-provider-credentials" pod "87e99792-812e-4038-8262-13eaf2a52347" (UID: "87e99792-812e-4038-8262-13eaf2a52347"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:38:49.249581 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.249460 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87e99792-812e-4038-8262-13eaf2a52347-kube-api-access-bdbpf" pod "87e99792-812e-4038-8262-13eaf2a52347" (UID: "87e99792-812e-4038-8262-13eaf2a52347"). InnerVolumeSpecName "kube-api-access-bdbpf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:38:49.332721 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.332686 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk"] Nov 04 07:38:49.333636 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.333611 5385 cpu_manager.go:414] "containerMap: removing container" podUID="87e99792-812e-4038-8262-13eaf2a52347" containerName="payload-processing" Nov 04 07:38:49.333636 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.333634 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="87e99792-812e-4038-8262-13eaf2a52347" containerName="payload-processing" Nov 04 07:38:49.333837 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.333801 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="87e99792-812e-4038-8262-13eaf2a52347" containerName="payload-processing" Nov 04 07:38:49.338539 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.338485 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:38:49.348932 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.348878 5385 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/87e99792-812e-4038-8262-13eaf2a52347-provider-credentials\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:38:49.348932 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.348909 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdbpf\" (UniqueName: \"kubernetes.io/projected/87e99792-812e-4038-8262-13eaf2a52347-kube-api-access-bdbpf\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:38:49.350595 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.350552 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-5c1e58-serving-cert\"" Nov 04 07:38:49.352388 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.351246 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk"] Nov 04 07:38:49.397685 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.397595 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:49.450413 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.450381 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3efd517-ed1d-4371-8009-856783d5a8c4-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk\" (UID: \"e3efd517-ed1d-4371-8009-856783d5a8c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:38:49.450609 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.450445 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvtbk\" (UniqueName: \"kubernetes.io/projected/e3efd517-ed1d-4371-8009-856783d5a8c4-kube-api-access-zvtbk\") pod \"maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk\" (UID: \"e3efd517-ed1d-4371-8009-856783d5a8c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:38:49.450609 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.450582 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3efd517-ed1d-4371-8009-856783d5a8c4-metrics-certs\") pod \"maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk\" (UID: \"e3efd517-ed1d-4371-8009-856783d5a8c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:38:49.450754 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.450736 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3efd517-ed1d-4371-8009-856783d5a8c4-maas-api-tls\") pod \"maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk\" (UID: \"e3efd517-ed1d-4371-8009-856783d5a8c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:38:49.510154 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.510114 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:49.521049 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.521015 5385 generic.go:356] "Generic (PLEG): container finished" podID="87e99792-812e-4038-8262-13eaf2a52347" containerID="46b14ffbe2fd6231f38019d565febbdeb6c0c380b3c6bce10525f4b4265f102a" exitCode=0 Nov 04 07:38:49.521204 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.521063 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" Nov 04 07:38:49.521204 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.521098 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" event={"ID":"87e99792-812e-4038-8262-13eaf2a52347","Type":"ContainerDied","Data":"46b14ffbe2fd6231f38019d565febbdeb6c0c380b3c6bce10525f4b4265f102a"} Nov 04 07:38:49.521204 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.521161 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd" event={"ID":"87e99792-812e-4038-8262-13eaf2a52347","Type":"ContainerDied","Data":"e9d4d3bf0e5e2e7e0cefcda2232f2a4f1b8258b9d50fec8b443ecfb8333acf94"} Nov 04 07:38:49.521204 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.521190 5385 scope.go:122] "RemoveContainer" containerID="46b14ffbe2fd6231f38019d565febbdeb6c0c380b3c6bce10525f4b4265f102a" Nov 04 07:38:49.534620 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.533352 5385 scope.go:122] "RemoveContainer" containerID="46b14ffbe2fd6231f38019d565febbdeb6c0c380b3c6bce10525f4b4265f102a" Nov 04 07:38:49.534620 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:38:49.533877 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46b14ffbe2fd6231f38019d565febbdeb6c0c380b3c6bce10525f4b4265f102a\": container with ID starting with 46b14ffbe2fd6231f38019d565febbdeb6c0c380b3c6bce10525f4b4265f102a not found: ID does not exist" containerID="46b14ffbe2fd6231f38019d565febbdeb6c0c380b3c6bce10525f4b4265f102a" Nov 04 07:38:49.534620 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.533912 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46b14ffbe2fd6231f38019d565febbdeb6c0c380b3c6bce10525f4b4265f102a"} err="failed to get container status \"46b14ffbe2fd6231f38019d565febbdeb6c0c380b3c6bce10525f4b4265f102a\": rpc error: code = NotFound desc = could not find container \"46b14ffbe2fd6231f38019d565febbdeb6c0c380b3c6bce10525f4b4265f102a\": container with ID starting with 46b14ffbe2fd6231f38019d565febbdeb6c0c380b3c6bce10525f4b4265f102a not found: ID does not exist" Nov 04 07:38:49.561254 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.558914 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvtbk\" (UniqueName: \"kubernetes.io/projected/e3efd517-ed1d-4371-8009-856783d5a8c4-kube-api-access-zvtbk\") pod \"maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk\" (UID: \"e3efd517-ed1d-4371-8009-856783d5a8c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:38:49.561254 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.558997 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3efd517-ed1d-4371-8009-856783d5a8c4-metrics-certs\") pod \"maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk\" (UID: \"e3efd517-ed1d-4371-8009-856783d5a8c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:38:49.561254 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.559115 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3efd517-ed1d-4371-8009-856783d5a8c4-maas-api-tls\") pod \"maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk\" (UID: \"e3efd517-ed1d-4371-8009-856783d5a8c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:38:49.561254 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.559287 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3efd517-ed1d-4371-8009-856783d5a8c4-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk\" (UID: \"e3efd517-ed1d-4371-8009-856783d5a8c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:38:49.561254 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.560319 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3efd517-ed1d-4371-8009-856783d5a8c4-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk\" (UID: \"e3efd517-ed1d-4371-8009-856783d5a8c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:38:49.568182 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.568116 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd"] Nov 04 07:38:49.573455 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.573398 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-646f5f8dd4-s2qzd"] Nov 04 07:38:49.574138 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.574082 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3efd517-ed1d-4371-8009-856783d5a8c4-metrics-certs\") pod \"maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk\" (UID: \"e3efd517-ed1d-4371-8009-856783d5a8c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:38:49.577146 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.577123 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3efd517-ed1d-4371-8009-856783d5a8c4-maas-api-tls\") pod \"maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk\" (UID: \"e3efd517-ed1d-4371-8009-856783d5a8c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:38:49.579289 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.579238 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvtbk\" (UniqueName: \"kubernetes.io/projected/e3efd517-ed1d-4371-8009-856783d5a8c4-kube-api-access-zvtbk\") pod \"maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk\" (UID: \"e3efd517-ed1d-4371-8009-856783d5a8c4\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:38:49.661569 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.661474 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:38:49.953536 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:49.953479 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk"] Nov 04 07:38:49.956085 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:38:49.956056 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3efd517_ed1d_4371_8009_856783d5a8c4.slice/crio-c75e41cae89a8662ffc2b7c2b723e177f14d4dea1fd1143acdb40825edf86cc3 WatchSource:0}: Error finding container c75e41cae89a8662ffc2b7c2b723e177f14d4dea1fd1143acdb40825edf86cc3: Status 404 returned error can't find the container with id c75e41cae89a8662ffc2b7c2b723e177f14d4dea1fd1143acdb40825edf86cc3 Nov 04 07:38:50.290946 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:50.290852 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:50.538267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:50.537325 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" event={"ID":"e3efd517-ed1d-4371-8009-856783d5a8c4","Type":"ContainerStarted","Data":"c75e41cae89a8662ffc2b7c2b723e177f14d4dea1fd1143acdb40825edf86cc3"} Nov 04 07:38:50.591940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:50.591906 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx"] Nov 04 07:38:50.622218 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:50.622185 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k"] Nov 04 07:38:50.708847 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:50.708814 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:50.814862 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:50.814778 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="87e99792-812e-4038-8262-13eaf2a52347" path="/var/lib/kubelet/pods/87e99792-812e-4038-8262-13eaf2a52347/volumes" Nov 04 07:38:51.237332 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:51.237282 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:51.337020 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:51.336968 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:51.546401 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:51.546316 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" event={"ID":"e3efd517-ed1d-4371-8009-856783d5a8c4","Type":"ContainerStarted","Data":"5f2ff44418df210448c75851cb09b0af0ef4791add0def682de86ce204a60cfe"} Nov 04 07:38:51.546554 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:51.546464 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:38:51.571574 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:51.571524 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" podStartSLOduration=2.148997371 podStartE2EDuration="2.571497785s" totalImagesPullingTime="422.500414ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:38:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:38:51.567692355 +0000 UTC m=+1366.901631939" watchObservedRunningTime="2026-11-04 07:38:51.571497785 +0000 UTC m=+1366.905437385" Nov 04 07:38:52.247731 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:52.247695 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:53.179435 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:53.179390 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:53.563933 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:53.563901 5385 generic.go:356] "Generic (PLEG): container finished" podID="e8a5b7b3-360d-475c-87b7-6a845394a09c" containerID="4d1b26e1e87ad26f5be3d994366c5f8a1ce778ffc5dbe2df5a0268842ef4c4f8" exitCode=0 Nov 04 07:38:53.564378 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:53.563953 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" event={"ID":"e8a5b7b3-360d-475c-87b7-6a845394a09c","Type":"ContainerDied","Data":"4d1b26e1e87ad26f5be3d994366c5f8a1ce778ffc5dbe2df5a0268842ef4c4f8"} Nov 04 07:38:53.564378 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:53.564368 5385 scope.go:122] "RemoveContainer" containerID="4d1b26e1e87ad26f5be3d994366c5f8a1ce778ffc5dbe2df5a0268842ef4c4f8" Nov 04 07:38:53.565981 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:53.565956 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:38:53.566052 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:53.566022 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:38:53.566089 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:53.566055 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:38:54.098033 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:54.097994 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:54.115726 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:54.115686 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:54.491216 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:54.491185 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:38:54.586395 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:54.586362 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" event={"ID":"e8a5b7b3-360d-475c-87b7-6a845394a09c","Type":"ContainerStarted","Data":"0d1de9dde96bc674c1447ee6124002858562b9ebf1c29bd9fc42c55b2cc9f476"} Nov 04 07:38:55.031741 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:55.031701 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:55.544023 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:55.543987 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:38:55.544998 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:55.544970 5385 patch_prober.go:29] interesting pod/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Nov 04 07:38:55.545114 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:55.545017 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" podUID="e8a5b7b3-360d-475c-87b7-6a845394a09c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Nov 04 07:38:55.895782 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:55.895748 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:56.544756 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:56.544724 5385 patch_prober.go:29] interesting pod/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Nov 04 07:38:56.544937 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:56.544772 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" podUID="e8a5b7b3-360d-475c-87b7-6a845394a09c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Nov 04 07:38:56.900587 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:56.900552 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:57.544413 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:57.544371 5385 patch_prober.go:29] interesting pod/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Nov 04 07:38:57.544626 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:57.544443 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" podUID="e8a5b7b3-360d-475c-87b7-6a845394a09c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Nov 04 07:38:57.575271 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:57.575241 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:38:57.971464 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:57.971422 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:58.124779 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:58.124728 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:58.242679 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:58.242597 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:58.547583 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:58.543783 5385 patch_prober.go:29] interesting pod/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Nov 04 07:38:58.547583 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:58.543839 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" podUID="e8a5b7b3-360d-475c-87b7-6a845394a09c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Nov 04 07:38:59.310046 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:59.310008 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:38:59.545902 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:59.545865 5385 patch_prober.go:29] interesting pod/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body= Nov 04 07:38:59.546225 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:59.546184 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" podUID="e8a5b7b3-360d-475c-87b7-6a845394a09c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" Nov 04 07:38:59.752028 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:38:59.751993 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:01.053432 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:39:01.053394 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:39:01.202890 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:01.202855 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:39:01.204252 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:01.204100 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:39:01.216014 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:01.214796 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:39:01.335604 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:01.328620 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:02.517874 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:02.517836 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:03.489767 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:03.489736 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:04.649472 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:04.649425 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:05.615575 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:05.615457 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" podUID="f4b66a64-9637-4b05-a20e-880ad2913b66" containerName="main" containerID="cri-o://91d05ad771bb658b6e4934cd24a8ccc92f332bbd998ca9487f6dfa06bebefa36" gracePeriod=45 Nov 04 07:39:05.646278 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:05.646244 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" podUID="85b100f6-ac81-414b-bd68-56276d8cd112" containerName="main" containerID="cri-o://8ce3de24efd0bf1fa7c2a41e9322243a5027ebe194c671441ec4380339b66d09" gracePeriod=45 Nov 04 07:39:05.760061 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:05.760021 5385 generic.go:356] "Generic (PLEG): container finished" podID="f4b66a64-9637-4b05-a20e-880ad2913b66" containerID="91d05ad771bb658b6e4934cd24a8ccc92f332bbd998ca9487f6dfa06bebefa36" exitCode=0 Nov 04 07:39:05.760575 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:05.760098 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" event={"ID":"f4b66a64-9637-4b05-a20e-880ad2913b66","Type":"ContainerDied","Data":"91d05ad771bb658b6e4934cd24a8ccc92f332bbd998ca9487f6dfa06bebefa36"} Nov 04 07:39:05.769769 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:05.769741 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:05.936602 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:05.936582 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:39:05.953390 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:05.953366 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:39:06.057551 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.056940 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4b66a64-9637-4b05-a20e-880ad2913b66-kube-api-access-sp6cq\" (UniqueName: \"kubernetes.io/projected/f4b66a64-9637-4b05-a20e-880ad2913b66-kube-api-access-sp6cq\") pod \"f4b66a64-9637-4b05-a20e-880ad2913b66\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " Nov 04 07:39:06.057551 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.057474 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-home\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-home\") pod \"f4b66a64-9637-4b05-a20e-880ad2913b66\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " Nov 04 07:39:06.057551 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.057545 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85b100f6-ac81-414b-bd68-56276d8cd112-tls-certs\" (UniqueName: \"kubernetes.io/secret/85b100f6-ac81-414b-bd68-56276d8cd112-tls-certs\") pod \"85b100f6-ac81-414b-bd68-56276d8cd112\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " Nov 04 07:39:06.057829 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.057614 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f4b66a64-9637-4b05-a20e-880ad2913b66-tls-certs\" (UniqueName: \"kubernetes.io/secret/f4b66a64-9637-4b05-a20e-880ad2913b66-tls-certs\") pod \"f4b66a64-9637-4b05-a20e-880ad2913b66\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " Nov 04 07:39:06.057829 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.057689 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-model-cache\") pod \"85b100f6-ac81-414b-bd68-56276d8cd112\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " Nov 04 07:39:06.057829 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.057720 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-dshm\") pod \"f4b66a64-9637-4b05-a20e-880ad2913b66\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " Nov 04 07:39:06.057829 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.057763 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-tmp-dir\") pod \"85b100f6-ac81-414b-bd68-56276d8cd112\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " Nov 04 07:39:06.057829 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.057804 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-tmp-dir\") pod \"f4b66a64-9637-4b05-a20e-880ad2913b66\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " Nov 04 07:39:06.058081 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.057913 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85b100f6-ac81-414b-bd68-56276d8cd112-kube-api-access-vqkq6\" (UniqueName: \"kubernetes.io/projected/85b100f6-ac81-414b-bd68-56276d8cd112-kube-api-access-vqkq6\") pod \"85b100f6-ac81-414b-bd68-56276d8cd112\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " Nov 04 07:39:06.058081 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.057977 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-home\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-home\") pod \"85b100f6-ac81-414b-bd68-56276d8cd112\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " Nov 04 07:39:06.058081 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.058007 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-model-cache\") pod \"f4b66a64-9637-4b05-a20e-880ad2913b66\" (UID: \"f4b66a64-9637-4b05-a20e-880ad2913b66\") " Nov 04 07:39:06.058081 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.058077 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-dshm\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-dshm\") pod \"85b100f6-ac81-414b-bd68-56276d8cd112\" (UID: \"85b100f6-ac81-414b-bd68-56276d8cd112\") " Nov 04 07:39:06.059545 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.058806 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-tmp-dir" pod "85b100f6-ac81-414b-bd68-56276d8cd112" (UID: "85b100f6-ac81-414b-bd68-56276d8cd112"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:06.059545 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.058996 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-home" pod "85b100f6-ac81-414b-bd68-56276d8cd112" (UID: "85b100f6-ac81-414b-bd68-56276d8cd112"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:06.059545 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.059204 5385 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-tmp-dir\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:06.059545 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.059202 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-model-cache" pod "f4b66a64-9637-4b05-a20e-880ad2913b66" (UID: "f4b66a64-9637-4b05-a20e-880ad2913b66"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:06.059545 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.059230 5385 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-home\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:06.060967 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.060745 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-home" pod "f4b66a64-9637-4b05-a20e-880ad2913b66" (UID: "f4b66a64-9637-4b05-a20e-880ad2913b66"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:06.060967 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.060956 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-tmp-dir" pod "f4b66a64-9637-4b05-a20e-880ad2913b66" (UID: "f4b66a64-9637-4b05-a20e-880ad2913b66"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:06.066270 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.063078 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-dshm" pod "f4b66a64-9637-4b05-a20e-880ad2913b66" (UID: "f4b66a64-9637-4b05-a20e-880ad2913b66"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:06.066270 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.063307 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-model-cache" pod "85b100f6-ac81-414b-bd68-56276d8cd112" (UID: "85b100f6-ac81-414b-bd68-56276d8cd112"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:06.067250 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.066981 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85b100f6-ac81-414b-bd68-56276d8cd112-tls-certs" pod "85b100f6-ac81-414b-bd68-56276d8cd112" (UID: "85b100f6-ac81-414b-bd68-56276d8cd112"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:39:06.067445 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.067424 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4b66a64-9637-4b05-a20e-880ad2913b66-tls-certs" pod "f4b66a64-9637-4b05-a20e-880ad2913b66" (UID: "f4b66a64-9637-4b05-a20e-880ad2913b66"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:39:06.072420 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.072375 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85b100f6-ac81-414b-bd68-56276d8cd112-kube-api-access-vqkq6" pod "85b100f6-ac81-414b-bd68-56276d8cd112" (UID: "85b100f6-ac81-414b-bd68-56276d8cd112"). InnerVolumeSpecName "kube-api-access-vqkq6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:39:06.074450 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.074417 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-dshm" pod "85b100f6-ac81-414b-bd68-56276d8cd112" (UID: "85b100f6-ac81-414b-bd68-56276d8cd112"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:06.077371 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.077340 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4b66a64-9637-4b05-a20e-880ad2913b66-kube-api-access-sp6cq" pod "f4b66a64-9637-4b05-a20e-880ad2913b66" (UID: "f4b66a64-9637-4b05-a20e-880ad2913b66"). InnerVolumeSpecName "kube-api-access-sp6cq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:39:06.159945 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.159909 5385 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4b66a64-9637-4b05-a20e-880ad2913b66-tls-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:06.159945 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.159936 5385 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-model-cache\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:06.160127 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.159950 5385 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-dshm\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:06.160127 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.159975 5385 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-tmp-dir\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:06.160127 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.159986 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqkq6\" (UniqueName: \"kubernetes.io/projected/85b100f6-ac81-414b-bd68-56276d8cd112-kube-api-access-vqkq6\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:06.160127 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.159996 5385 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-model-cache\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:06.160127 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.160004 5385 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/85b100f6-ac81-414b-bd68-56276d8cd112-dshm\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:06.160127 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.160013 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sp6cq\" (UniqueName: \"kubernetes.io/projected/f4b66a64-9637-4b05-a20e-880ad2913b66-kube-api-access-sp6cq\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:06.160127 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.160020 5385 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f4b66a64-9637-4b05-a20e-880ad2913b66-home\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:06.160127 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.160028 5385 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/85b100f6-ac81-414b-bd68-56276d8cd112-tls-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:06.596672 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.596626 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:06.767128 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.767077 5385 generic.go:356] "Generic (PLEG): container finished" podID="85b100f6-ac81-414b-bd68-56276d8cd112" containerID="8ce3de24efd0bf1fa7c2a41e9322243a5027ebe194c671441ec4380339b66d09" exitCode=0 Nov 04 07:39:06.767616 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.767139 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" Nov 04 07:39:06.767616 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.767157 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" event={"ID":"85b100f6-ac81-414b-bd68-56276d8cd112","Type":"ContainerDied","Data":"8ce3de24efd0bf1fa7c2a41e9322243a5027ebe194c671441ec4380339b66d09"} Nov 04 07:39:06.767616 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.767205 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k" event={"ID":"85b100f6-ac81-414b-bd68-56276d8cd112","Type":"ContainerDied","Data":"250dae6b8fb48cf04f2df0736680c42c34ff6e7568d6c2b1e3edc44747f20f8e"} Nov 04 07:39:06.767616 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.767230 5385 scope.go:122] "RemoveContainer" containerID="8ce3de24efd0bf1fa7c2a41e9322243a5027ebe194c671441ec4380339b66d09" Nov 04 07:39:06.768716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.768701 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" Nov 04 07:39:06.768797 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.768733 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx" event={"ID":"f4b66a64-9637-4b05-a20e-880ad2913b66","Type":"ContainerDied","Data":"441c79a4075d8be2adbc36609fd6b32a3e3c702937174cad6dc82e9d20020fc2"} Nov 04 07:39:06.777928 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.777904 5385 scope.go:122] "RemoveContainer" containerID="8ce3de24efd0bf1fa7c2a41e9322243a5027ebe194c671441ec4380339b66d09" Nov 04 07:39:06.778272 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:39:06.778179 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ce3de24efd0bf1fa7c2a41e9322243a5027ebe194c671441ec4380339b66d09\": container with ID starting with 8ce3de24efd0bf1fa7c2a41e9322243a5027ebe194c671441ec4380339b66d09 not found: ID does not exist" containerID="8ce3de24efd0bf1fa7c2a41e9322243a5027ebe194c671441ec4380339b66d09" Nov 04 07:39:06.778272 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.778206 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ce3de24efd0bf1fa7c2a41e9322243a5027ebe194c671441ec4380339b66d09"} err="failed to get container status \"8ce3de24efd0bf1fa7c2a41e9322243a5027ebe194c671441ec4380339b66d09\": rpc error: code = NotFound desc = could not find container \"8ce3de24efd0bf1fa7c2a41e9322243a5027ebe194c671441ec4380339b66d09\": container with ID starting with 8ce3de24efd0bf1fa7c2a41e9322243a5027ebe194c671441ec4380339b66d09 not found: ID does not exist" Nov 04 07:39:06.778272 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.778222 5385 scope.go:122] "RemoveContainer" containerID="91d05ad771bb658b6e4934cd24a8ccc92f332bbd998ca9487f6dfa06bebefa36" Nov 04 07:39:06.834132 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.834110 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx"] Nov 04 07:39:06.885330 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.885307 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/distinct-2-w3-60ed4a-kserve-7c56ff9f57-rf4gx"] Nov 04 07:39:06.912418 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.912391 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k"] Nov 04 07:39:06.943216 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:06.943190 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-60ed4a/simulator-w3-60ed4a-kserve-b9684f49d-8sm6k"] Nov 04 07:39:07.486564 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:07.484285 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:08.198422 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:08.198383 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:08.550325 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:08.550256 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:08.865571 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:08.856072 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85b100f6-ac81-414b-bd68-56276d8cd112" path="/var/lib/kubelet/pods/85b100f6-ac81-414b-bd68-56276d8cd112/volumes" Nov 04 07:39:08.865571 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:08.856796 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4b66a64-9637-4b05-a20e-880ad2913b66" path="/var/lib/kubelet/pods/f4b66a64-9637-4b05-a20e-880ad2913b66/volumes" Nov 04 07:39:08.865571 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:08.857187 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz"] Nov 04 07:39:08.865571 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:08.858047 5385 cpu_manager.go:414] "containerMap: removing container" podUID="f4b66a64-9637-4b05-a20e-880ad2913b66" containerName="main" Nov 04 07:39:08.865571 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:08.858062 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4b66a64-9637-4b05-a20e-880ad2913b66" containerName="main" Nov 04 07:39:08.865571 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:08.858103 5385 cpu_manager.go:414] "containerMap: removing container" podUID="85b100f6-ac81-414b-bd68-56276d8cd112" containerName="main" Nov 04 07:39:08.865571 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:08.858110 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85b100f6-ac81-414b-bd68-56276d8cd112" containerName="main" Nov 04 07:39:08.865571 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:08.858247 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4b66a64-9637-4b05-a20e-880ad2913b66" containerName="main" Nov 04 07:39:08.865571 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:08.858264 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="85b100f6-ac81-414b-bd68-56276d8cd112" containerName="main" Nov 04 07:39:08.865571 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:08.864743 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:08.882080 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:08.880964 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz"] Nov 04 07:39:08.891415 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:08.891392 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-a558a6-openshift-default-dockercfg-b9bch\"" Nov 04 07:39:08.973838 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:08.971880 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:09.012037 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.011993 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-token\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.012241 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.012058 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-data\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.012241 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.012180 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-podinfo\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.012350 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.012265 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-workload-certs\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.012350 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.012291 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-envoy\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.012350 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.012338 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-credential-socket\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.012482 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.012386 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63ab0edc-d9ba-486d-bfc0-7060678daea2-istiod-ca-cert\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.012482 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.012420 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-workload-socket\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.012482 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.012475 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d75sg\" (UniqueName: \"kubernetes.io/projected/63ab0edc-d9ba-486d-bfc0-7060678daea2-kube-api-access-d75sg\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.113721 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.113683 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-token\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.113915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.113739 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-data\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.113915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.113835 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-podinfo\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.113915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.113882 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-workload-certs\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.113915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.113900 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-envoy\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.114150 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.113933 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-credential-socket\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.114150 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.113967 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63ab0edc-d9ba-486d-bfc0-7060678daea2-istiod-ca-cert\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.114150 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.113988 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-workload-socket\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.114150 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.114026 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d75sg\" (UniqueName: \"kubernetes.io/projected/63ab0edc-d9ba-486d-bfc0-7060678daea2-kube-api-access-d75sg\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.114318 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.114152 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-data\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.114403 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.114381 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-credential-socket\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.114463 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.114433 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-workload-socket\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.114582 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.114560 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-workload-certs\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.114706 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.114689 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63ab0edc-d9ba-486d-bfc0-7060678daea2-istiod-ca-cert\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.116300 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.116246 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-envoy\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.116659 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.116641 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-podinfo\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.132430 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.132400 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d75sg\" (UniqueName: \"kubernetes.io/projected/63ab0edc-d9ba-486d-bfc0-7060678daea2-kube-api-access-d75sg\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.132588 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.132479 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-token\") pod \"e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.198672 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.198640 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:09.372151 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.372119 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:09.495235 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.495209 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz"] Nov 04 07:39:09.495764 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:39:09.495737 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63ab0edc_d9ba_486d_bfc0_7060678daea2.slice/crio-de2071b7c1906f2a02f052588024bcf3ffaba16d19508b3f8f71a0d065409dfb WatchSource:0}: Error finding container de2071b7c1906f2a02f052588024bcf3ffaba16d19508b3f8f71a0d065409dfb: Status 404 returned error can't find the container with id de2071b7c1906f2a02f052588024bcf3ffaba16d19508b3f8f71a0d065409dfb Nov 04 07:39:09.498455 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.498419 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:39:09.498588 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.498500 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:39:09.498588 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.498566 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:39:09.556343 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.556314 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:09.853585 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.853484 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" event={"ID":"63ab0edc-d9ba-486d-bfc0-7060678daea2","Type":"ContainerStarted","Data":"19a36bedda2349d3371d0983319133ed846dd32d1e66982ff1cb0c7af13512f2"} Nov 04 07:39:09.853585 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.853562 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" event={"ID":"63ab0edc-d9ba-486d-bfc0-7060678daea2","Type":"ContainerStarted","Data":"de2071b7c1906f2a02f052588024bcf3ffaba16d19508b3f8f71a0d065409dfb"} Nov 04 07:39:09.882347 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:09.882294 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" podStartSLOduration=1.882281196 podStartE2EDuration="1.882281196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:39:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:39:09.879870727 +0000 UTC m=+1385.213810311" watchObservedRunningTime="2026-11-04 07:39:09.882281196 +0000 UTC m=+1385.216220780" Nov 04 07:39:10.199421 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:10.199392 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:10.205276 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:10.205253 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:10.700903 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:10.700868 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:10.858705 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:10.858666 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:10.859463 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:10.859440 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:12.398530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:12.398413 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:12.610971 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:12.610928 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:12.760874 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:12.760795 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:13.671630 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:13.670645 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:14.300872 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.300832 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:14.428604 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.428567 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj"] Nov 04 07:39:14.432917 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.432895 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.617589 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.597959 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:14.667219 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.667183 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj"] Nov 04 07:39:14.668362 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.668331 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.668693 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.668668 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.668855 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.668836 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbpg2\" (UniqueName: \"kubernetes.io/projected/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-kube-api-access-gbpg2\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.668993 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.668975 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.669130 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.669114 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.669312 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.669294 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.669537 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.669498 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.670764 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.670743 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.671147 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.671127 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.671364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.670649 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp"] Nov 04 07:39:14.676757 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.676735 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.679775 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.679751 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp"] Nov 04 07:39:14.772168 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772136 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.772168 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772169 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.772341 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772190 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb02a450-1430-46b8-95bd-94edf3e27328-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.772341 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772284 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.772433 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772376 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.772480 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772454 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.772563 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772487 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb02a450-1430-46b8-95bd-94edf3e27328-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.772563 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772555 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb02a450-1430-46b8-95bd-94edf3e27328-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.772677 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772651 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.772728 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772683 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbpg2\" (UniqueName: \"kubernetes.io/projected/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-kube-api-access-gbpg2\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.772728 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772705 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.772827 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772734 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.772827 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772754 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.772827 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772788 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.772983 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772828 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.772983 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772922 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.772983 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772960 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.773192 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.772989 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.773192 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.773024 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.773192 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.773062 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74vbq\" (UniqueName: \"kubernetes.io/projected/cb02a450-1430-46b8-95bd-94edf3e27328-kube-api-access-74vbq\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.773192 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.773064 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.773419 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.773318 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.773419 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.773372 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.774922 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.774901 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.775166 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.775149 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.794718 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.794690 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.794854 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.794827 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbpg2\" (UniqueName: \"kubernetes.io/projected/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-kube-api-access-gbpg2\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:14.874214 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.874124 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.874214 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.874168 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74vbq\" (UniqueName: \"kubernetes.io/projected/cb02a450-1430-46b8-95bd-94edf3e27328-kube-api-access-74vbq\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.874431 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.874232 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.874431 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.874255 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.874431 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.874286 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb02a450-1430-46b8-95bd-94edf3e27328-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.874431 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.874324 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.874681 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.874496 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb02a450-1430-46b8-95bd-94edf3e27328-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.874681 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.874588 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb02a450-1430-46b8-95bd-94edf3e27328-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.874681 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.874627 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-workload-certs\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.874681 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.874664 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-workload-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.874885 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.874733 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-credential-socket\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.874885 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.874750 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.875028 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.875007 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-istio-data\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.875218 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.875201 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb02a450-1430-46b8-95bd-94edf3e27328-istiod-ca-cert\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.876970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.876945 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb02a450-1430-46b8-95bd-94edf3e27328-istio-podinfo\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.876970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.876959 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-istio-envoy\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.892992 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.892967 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-74vbq\" (UniqueName: \"kubernetes.io/projected/cb02a450-1430-46b8-95bd-94edf3e27328-kube-api-access-74vbq\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.893827 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.893801 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb02a450-1430-46b8-95bd-94edf3e27328-istio-token\") pod \"e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:14.992069 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:14.992034 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:15.049115 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:15.048928 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:15.305955 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:15.305927 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp"] Nov 04 07:39:15.307179 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:39:15.307150 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb02a450_1430_46b8_95bd_94edf3e27328.slice/crio-068efed73ad911d0b8376399752358453a874078a69aec10649a07d9064388cc WatchSource:0}: Error finding container 068efed73ad911d0b8376399752358453a874078a69aec10649a07d9064388cc: Status 404 returned error can't find the container with id 068efed73ad911d0b8376399752358453a874078a69aec10649a07d9064388cc Nov 04 07:39:15.309785 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:15.309754 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:39:15.309876 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:15.309815 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:39:15.309876 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:15.309847 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:39:15.357881 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:15.357856 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj"] Nov 04 07:39:15.358660 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:39:15.358636 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode977f4f4_c3aa_484d_b64a_fda4d19ce7de.slice/crio-d8034395390716756480fd4aad5bf753b4370e177619c104703bc867399fb049 WatchSource:0}: Error finding container d8034395390716756480fd4aad5bf753b4370e177619c104703bc867399fb049: Status 404 returned error can't find the container with id d8034395390716756480fd4aad5bf753b4370e177619c104703bc867399fb049 Nov 04 07:39:15.361364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:15.361335 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:39:15.361448 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:15.361393 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:39:15.361448 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:15.361422 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:39:15.494527 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:15.492382 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:15.899189 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:15.899145 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" event={"ID":"e977f4f4-c3aa-484d-b64a-fda4d19ce7de","Type":"ContainerStarted","Data":"7904b10e7609f2deaa622d030e6cd8627ff930057896fe0e905d8ebdd8ec0c31"} Nov 04 07:39:15.899702 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:15.899218 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" event={"ID":"e977f4f4-c3aa-484d-b64a-fda4d19ce7de","Type":"ContainerStarted","Data":"d8034395390716756480fd4aad5bf753b4370e177619c104703bc867399fb049"} Nov 04 07:39:15.901019 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:15.900993 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" event={"ID":"cb02a450-1430-46b8-95bd-94edf3e27328","Type":"ContainerStarted","Data":"34e23a4b4f5503b92d5c7289b953e6619a4c133323ebeb62f94b23a9788edc7a"} Nov 04 07:39:15.901125 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:15.901033 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" event={"ID":"cb02a450-1430-46b8-95bd-94edf3e27328","Type":"ContainerStarted","Data":"068efed73ad911d0b8376399752358453a874078a69aec10649a07d9064388cc"} Nov 04 07:39:15.927428 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:15.927374 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" podStartSLOduration=1.92735867 podStartE2EDuration="1.92735867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:39:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:39:15.923912047 +0000 UTC m=+1391.257851657" watchObservedRunningTime="2026-11-04 07:39:15.92735867 +0000 UTC m=+1391.261298322" Nov 04 07:39:15.957400 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:15.957341 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" podStartSLOduration=1.95732716 podStartE2EDuration="1.95732716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:39:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:39:15.956977631 +0000 UTC m=+1391.290917242" watchObservedRunningTime="2026-11-04 07:39:15.95732716 +0000 UTC m=+1391.291266744" Nov 04 07:39:15.994529 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:15.993607 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:16.052069 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:16.051859 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:16.072526 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:16.060891 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Nov 04 07:39:16.072526 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:16.060952 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" podUID="e977f4f4-c3aa-484d-b64a-fda4d19ce7de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Nov 04 07:39:16.508515 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:16.508478 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:16.737042 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:16.736978 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:17.000024 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:16.999982 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Nov 04 07:39:17.000533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:17.000040 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" podUID="cb02a450-1430-46b8-95bd-94edf3e27328" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Nov 04 07:39:17.067531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:17.053845 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Nov 04 07:39:17.067531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:17.053900 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" podUID="e977f4f4-c3aa-484d-b64a-fda4d19ce7de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Nov 04 07:39:17.902410 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:17.902355 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:17.994532 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:17.993967 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Nov 04 07:39:17.994532 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:17.994025 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" podUID="cb02a450-1430-46b8-95bd-94edf3e27328" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Nov 04 07:39:18.053532 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:18.049869 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Nov 04 07:39:18.053532 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:18.050021 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" podUID="e977f4f4-c3aa-484d-b64a-fda4d19ce7de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Nov 04 07:39:18.091548 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:18.089486 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:18.993532 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:18.992888 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Nov 04 07:39:18.993532 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:18.992952 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" podUID="cb02a450-1430-46b8-95bd-94edf3e27328" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Nov 04 07:39:19.056529 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:19.052829 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Nov 04 07:39:19.056529 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:19.052888 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" podUID="e977f4f4-c3aa-484d-b64a-fda4d19ce7de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Nov 04 07:39:19.609528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:19.605734 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:20.006736 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:20.006699 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Nov 04 07:39:20.006994 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:20.006965 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" podUID="cb02a450-1430-46b8-95bd-94edf3e27328" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Nov 04 07:39:20.069100 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:20.069068 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Nov 04 07:39:20.069694 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:20.069662 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" podUID="e977f4f4-c3aa-484d-b64a-fda4d19ce7de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Nov 04 07:39:20.907552 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:20.907516 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:20.992955 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:20.992916 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Nov 04 07:39:20.993207 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:20.993176 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" podUID="cb02a450-1430-46b8-95bd-94edf3e27328" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Nov 04 07:39:21.055529 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:21.052848 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Nov 04 07:39:21.055529 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:21.052906 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" podUID="e977f4f4-c3aa-484d-b64a-fda4d19ce7de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Nov 04 07:39:21.992520 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:21.992473 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" start-of-body= Nov 04 07:39:21.993017 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:21.992574 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" podUID="cb02a450-1430-46b8-95bd-94edf3e27328" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.77:15021/healthz/ready\": dial tcp 10.133.0.77:15021: connect: connection refused" Nov 04 07:39:22.061324 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:22.061288 5385 patch_prober.go:29] interesting pod/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Nov 04 07:39:22.061595 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:22.061564 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" podUID="e977f4f4-c3aa-484d-b64a-fda4d19ce7de" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Nov 04 07:39:22.081727 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:22.081669 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:23.052110 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:23.052006 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:23.139145 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:23.139101 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:23.140935 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:23.140910 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:23.143019 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:23.142992 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:23.172853 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:23.172821 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:23.174673 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:23.174650 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:23.192807 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:23.192775 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:24.318036 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:24.318001 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h"] Nov 04 07:39:24.318596 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:24.318210 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" podUID="a34590d5-f58f-4073-b0fc-05ae69dc8e66" containerName="maas-api" containerID="cri-o://ecc71b93856e6f607c2a1e1e0584893463dc953d1f5c645437485907a6392b53" gracePeriod=30 Nov 04 07:39:24.379348 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:24.379313 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:24.756555 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:24.752314 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:39:24.797256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:24.797218 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a34590d5-f58f-4073-b0fc-05ae69dc8e66-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a34590d5-f58f-4073-b0fc-05ae69dc8e66-openshift-service-ca\") pod \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\" (UID: \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\") " Nov 04 07:39:24.797546 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:24.797528 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a34590d5-f58f-4073-b0fc-05ae69dc8e66-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a34590d5-f58f-4073-b0fc-05ae69dc8e66-metrics-certs\") pod \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\" (UID: \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\") " Nov 04 07:39:24.797743 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:24.797727 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a34590d5-f58f-4073-b0fc-05ae69dc8e66-kube-api-access-g65sm\" (UniqueName: \"kubernetes.io/projected/a34590d5-f58f-4073-b0fc-05ae69dc8e66-kube-api-access-g65sm\") pod \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\" (UID: \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\") " Nov 04 07:39:24.797935 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:24.797922 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a34590d5-f58f-4073-b0fc-05ae69dc8e66-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a34590d5-f58f-4073-b0fc-05ae69dc8e66-maas-api-tls\") pod \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\" (UID: \"a34590d5-f58f-4073-b0fc-05ae69dc8e66\") " Nov 04 07:39:24.807895 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:24.807857 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a34590d5-f58f-4073-b0fc-05ae69dc8e66-openshift-service-ca" pod "a34590d5-f58f-4073-b0fc-05ae69dc8e66" (UID: "a34590d5-f58f-4073-b0fc-05ae69dc8e66"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:39:24.844825 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:24.833042 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a34590d5-f58f-4073-b0fc-05ae69dc8e66-maas-api-tls" pod "a34590d5-f58f-4073-b0fc-05ae69dc8e66" (UID: "a34590d5-f58f-4073-b0fc-05ae69dc8e66"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:39:24.849567 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:24.849028 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a34590d5-f58f-4073-b0fc-05ae69dc8e66-metrics-certs" pod "a34590d5-f58f-4073-b0fc-05ae69dc8e66" (UID: "a34590d5-f58f-4073-b0fc-05ae69dc8e66"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:39:24.861976 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:24.861934 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a34590d5-f58f-4073-b0fc-05ae69dc8e66-kube-api-access-g65sm" pod "a34590d5-f58f-4073-b0fc-05ae69dc8e66" (UID: "a34590d5-f58f-4073-b0fc-05ae69dc8e66"). InnerVolumeSpecName "kube-api-access-g65sm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:39:24.869315 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:24.869281 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:24.902838 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:24.902800 5385 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a34590d5-f58f-4073-b0fc-05ae69dc8e66-maas-api-tls\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:24.902838 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:24.902832 5385 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a34590d5-f58f-4073-b0fc-05ae69dc8e66-openshift-service-ca\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:24.902838 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:24.902841 5385 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a34590d5-f58f-4073-b0fc-05ae69dc8e66-metrics-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:24.902838 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:24.902850 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g65sm\" (UniqueName: \"kubernetes.io/projected/a34590d5-f58f-4073-b0fc-05ae69dc8e66-kube-api-access-g65sm\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:25.097320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:25.097207 5385 generic.go:356] "Generic (PLEG): container finished" podID="a34590d5-f58f-4073-b0fc-05ae69dc8e66" containerID="ecc71b93856e6f607c2a1e1e0584893463dc953d1f5c645437485907a6392b53" exitCode=0 Nov 04 07:39:25.097523 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:25.097361 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" Nov 04 07:39:25.097523 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:25.097452 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" event={"ID":"a34590d5-f58f-4073-b0fc-05ae69dc8e66","Type":"ContainerDied","Data":"ecc71b93856e6f607c2a1e1e0584893463dc953d1f5c645437485907a6392b53"} Nov 04 07:39:25.097675 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:25.097557 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h" event={"ID":"a34590d5-f58f-4073-b0fc-05ae69dc8e66","Type":"ContainerDied","Data":"0eb64664c33709509396a699c427e0244af82e5382adea9dd202f1011d4c38a5"} Nov 04 07:39:25.097675 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:25.097599 5385 scope.go:122] "RemoveContainer" containerID="ecc71b93856e6f607c2a1e1e0584893463dc953d1f5c645437485907a6392b53" Nov 04 07:39:25.110444 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:25.110347 5385 scope.go:122] "RemoveContainer" containerID="ecc71b93856e6f607c2a1e1e0584893463dc953d1f5c645437485907a6392b53" Nov 04 07:39:25.110745 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:39:25.110716 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecc71b93856e6f607c2a1e1e0584893463dc953d1f5c645437485907a6392b53\": container with ID starting with ecc71b93856e6f607c2a1e1e0584893463dc953d1f5c645437485907a6392b53 not found: ID does not exist" containerID="ecc71b93856e6f607c2a1e1e0584893463dc953d1f5c645437485907a6392b53" Nov 04 07:39:25.110860 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:25.110756 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecc71b93856e6f607c2a1e1e0584893463dc953d1f5c645437485907a6392b53"} err="failed to get container status \"ecc71b93856e6f607c2a1e1e0584893463dc953d1f5c645437485907a6392b53\": rpc error: code = NotFound desc = could not find container \"ecc71b93856e6f607c2a1e1e0584893463dc953d1f5c645437485907a6392b53\": container with ID starting with ecc71b93856e6f607c2a1e1e0584893463dc953d1f5c645437485907a6392b53 not found: ID does not exist" Nov 04 07:39:25.157333 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:25.157300 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h"] Nov 04 07:39:25.162875 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:25.162775 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-60ed4a-7b497c9c9d-lt94h"] Nov 04 07:39:26.303548 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:26.297073 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:26.813738 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:26.813706 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a34590d5-f58f-4073-b0fc-05ae69dc8e66" path="/var/lib/kubelet/pods/a34590d5-f58f-4073-b0fc-05ae69dc8e66/volumes" Nov 04 07:39:27.557528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:27.556632 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:28.375264 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.375228 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p"] Nov 04 07:39:28.376609 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.376582 5385 cpu_manager.go:414] "containerMap: removing container" podUID="a34590d5-f58f-4073-b0fc-05ae69dc8e66" containerName="maas-api" Nov 04 07:39:28.376782 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.376750 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a34590d5-f58f-4073-b0fc-05ae69dc8e66" containerName="maas-api" Nov 04 07:39:28.377052 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.376986 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="a34590d5-f58f-4073-b0fc-05ae69dc8e66" containerName="maas-api" Nov 04 07:39:28.395576 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.395544 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p"] Nov 04 07:39:28.395762 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.395674 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:39:28.406996 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.406970 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-a558a6-serving-cert\"" Nov 04 07:39:28.493569 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.493167 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/194ab3ed-32ab-4483-b60c-1c3c02451ee2-maas-api-tls\") pod \"maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p\" (UID: \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:39:28.493569 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.493303 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/194ab3ed-32ab-4483-b60c-1c3c02451ee2-metrics-certs\") pod \"maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p\" (UID: \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:39:28.493569 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.493536 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7h79\" (UniqueName: \"kubernetes.io/projected/194ab3ed-32ab-4483-b60c-1c3c02451ee2-kube-api-access-p7h79\") pod \"maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p\" (UID: \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:39:28.493569 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.493567 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/194ab3ed-32ab-4483-b60c-1c3c02451ee2-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p\" (UID: \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:39:28.519058 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.518968 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:28.594348 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.594313 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7h79\" (UniqueName: \"kubernetes.io/projected/194ab3ed-32ab-4483-b60c-1c3c02451ee2-kube-api-access-p7h79\") pod \"maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p\" (UID: \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:39:28.594348 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.594354 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/194ab3ed-32ab-4483-b60c-1c3c02451ee2-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p\" (UID: \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:39:28.594862 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.594400 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/194ab3ed-32ab-4483-b60c-1c3c02451ee2-maas-api-tls\") pod \"maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p\" (UID: \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:39:28.594862 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.594467 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/194ab3ed-32ab-4483-b60c-1c3c02451ee2-metrics-certs\") pod \"maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p\" (UID: \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:39:28.595540 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.595482 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/194ab3ed-32ab-4483-b60c-1c3c02451ee2-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p\" (UID: \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:39:28.597164 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.597131 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/194ab3ed-32ab-4483-b60c-1c3c02451ee2-metrics-certs\") pod \"maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p\" (UID: \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:39:28.597263 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.597236 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/194ab3ed-32ab-4483-b60c-1c3c02451ee2-maas-api-tls\") pod \"maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p\" (UID: \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:39:28.608149 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.608105 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7h79\" (UniqueName: \"kubernetes.io/projected/194ab3ed-32ab-4483-b60c-1c3c02451ee2-kube-api-access-p7h79\") pod \"maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p\" (UID: \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:39:28.652048 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.651969 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:28.716857 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.716820 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:39:28.911717 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:28.911638 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:29.128396 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:29.128363 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p"] Nov 04 07:39:29.173267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:29.173172 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" event={"ID":"194ab3ed-32ab-4483-b60c-1c3c02451ee2","Type":"ContainerStarted","Data":"03dc4490005f540925123bc8dd7127fa9be8390b1ff56b6779b4cce6d15528e5"} Nov 04 07:39:29.297478 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:29.297433 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:29.450223 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:29.450118 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:29.646991 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:29.646945 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:30.126630 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:30.126592 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:30.223650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:30.213465 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" event={"ID":"194ab3ed-32ab-4483-b60c-1c3c02451ee2","Type":"ContainerStarted","Data":"6dded2510d6a510ee9aa300cc311e76564a459476ecf6dda07f4d247a71a0a14"} Nov 04 07:39:30.223650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:30.214594 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:39:30.253814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:30.253754 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" podStartSLOduration=1.566216463 podStartE2EDuration="2.253733434s" totalImagesPullingTime="687.516971ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:39:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:39:30.252923068 +0000 UTC m=+1405.586862656" watchObservedRunningTime="2026-11-04 07:39:30.253733434 +0000 UTC m=+1405.587673019" Nov 04 07:39:30.874422 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:39:30.874386 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:39:31.263844 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:31.263753 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:31.880671 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:31.880639 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:32.223966 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:32.223866 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:33.322200 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:33.322166 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:34.203741 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:34.203710 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:35.110288 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:35.110254 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:36.408833 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:36.401126 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:37.232411 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:37.232382 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:39:37.326384 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:37.326342 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:37.846270 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:37.846229 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:38.160355 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:38.160320 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:38.473017 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:38.472936 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:38.616025 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:38.615943 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:38.900615 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:38.900572 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp"] Nov 04 07:39:38.901418 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:38.901362 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" podUID="cb02a450-1430-46b8-95bd-94edf3e27328" containerName="istio-proxy" containerID="cri-o://34e23a4b4f5503b92d5c7289b953e6619a4c133323ebeb62f94b23a9788edc7a" gracePeriod=30 Nov 04 07:39:38.914292 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:38.909942 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj"] Nov 04 07:39:38.919651 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:38.918838 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9"] Nov 04 07:39:38.923343 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:38.923309 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" podUID="6b7e3053-63a4-43c2-ada7-f98132ef6d4d" containerName="istio-proxy" containerID="cri-o://eebcb0d628d1dcab4f6f6c9d75597c9ea7361a8bf0e638cf610f53c36d6ac64a" gracePeriod=30 Nov 04 07:39:38.927392 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:38.926559 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" podUID="e977f4f4-c3aa-484d-b64a-fda4d19ce7de" containerName="istio-proxy" containerID="cri-o://7904b10e7609f2deaa622d030e6cd8627ff930057896fe0e905d8ebdd8ec0c31" gracePeriod=30 Nov 04 07:39:38.980017 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:38.978062 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:39.945252 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:39.945220 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:40.370548 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:40.370482 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:40.472083 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:40.472041 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:40.911381 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:40.911343 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:41.989880 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:41.989837 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:42.835271 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:42.835236 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:44.057388 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.057297 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:44.170291 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.170065 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:44.294570 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.287108 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:44.480830 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.473045 5385 generic.go:356] "Generic (PLEG): container finished" podID="e977f4f4-c3aa-484d-b64a-fda4d19ce7de" containerID="7904b10e7609f2deaa622d030e6cd8627ff930057896fe0e905d8ebdd8ec0c31" exitCode=0 Nov 04 07:39:44.480830 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.473241 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" event={"ID":"e977f4f4-c3aa-484d-b64a-fda4d19ce7de","Type":"ContainerDied","Data":"7904b10e7609f2deaa622d030e6cd8627ff930057896fe0e905d8ebdd8ec0c31"} Nov 04 07:39:44.556530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.554324 5385 generic.go:356] "Generic (PLEG): container finished" podID="cb02a450-1430-46b8-95bd-94edf3e27328" containerID="34e23a4b4f5503b92d5c7289b953e6619a4c133323ebeb62f94b23a9788edc7a" exitCode=0 Nov 04 07:39:44.556530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.554435 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" event={"ID":"cb02a450-1430-46b8-95bd-94edf3e27328","Type":"ContainerDied","Data":"34e23a4b4f5503b92d5c7289b953e6619a4c133323ebeb62f94b23a9788edc7a"} Nov 04 07:39:44.567537 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.560221 5385 generic.go:356] "Generic (PLEG): container finished" podID="6b7e3053-63a4-43c2-ada7-f98132ef6d4d" containerID="eebcb0d628d1dcab4f6f6c9d75597c9ea7361a8bf0e638cf610f53c36d6ac64a" exitCode=0 Nov 04 07:39:44.567537 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.560294 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" event={"ID":"6b7e3053-63a4-43c2-ada7-f98132ef6d4d","Type":"ContainerDied","Data":"eebcb0d628d1dcab4f6f6c9d75597c9ea7361a8bf0e638cf610f53c36d6ac64a"} Nov 04 07:39:44.567537 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.560366 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" event={"ID":"6b7e3053-63a4-43c2-ada7-f98132ef6d4d","Type":"ContainerDied","Data":"c0c1f35b4c932eec263f7d30ef2683d5c421f8aa830af133008de4d53f5422d1"} Nov 04 07:39:44.567537 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.560392 5385 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0c1f35b4c932eec263f7d30ef2683d5c421f8aa830af133008de4d53f5422d1" Nov 04 07:39:44.567537 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.560512 5385 scope.go:122] "RemoveContainer" containerID="05a388de22fde529f354962c82f3dfe918023fd743d50c458d4f22b225143b5f" Nov 04 07:39:44.611462 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.609815 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:39:44.715581 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.715555 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:44.719071 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.719050 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:44.734655 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.734597 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.758535 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-credential-socket" pod "6b7e3053-63a4-43c2-ada7-f98132ef6d4d" (UID: "6b7e3053-63a4-43c2-ada7-f98132ef6d4d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.758682 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-credential-socket\") pod \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.758754 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-token\" (UniqueName: \"kubernetes.io/projected/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-token\") pod \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.759093 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-workload-socket" pod "6b7e3053-63a4-43c2-ada7-f98132ef6d4d" (UID: "6b7e3053-63a4-43c2-ada7-f98132ef6d4d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.759537 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-workload-socket\") pod \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.759593 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-data\") pod \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.759696 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istiod-ca-cert\") pod \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.759741 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-kube-api-access-8xwt2\" (UniqueName: \"kubernetes.io/projected/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-kube-api-access-8xwt2\") pod \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.759771 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-envoy\") pod \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.759943 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-podinfo\") pod \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.759999 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-workload-certs\") pod \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\" (UID: \"6b7e3053-63a4-43c2-ada7-f98132ef6d4d\") " Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.760931 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istiod-ca-cert" pod "6b7e3053-63a4-43c2-ada7-f98132ef6d4d" (UID: "6b7e3053-63a4-43c2-ada7-f98132ef6d4d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.761279 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-data" pod "6b7e3053-63a4-43c2-ada7-f98132ef6d4d" (UID: "6b7e3053-63a4-43c2-ada7-f98132ef6d4d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.761724 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-workload-certs" pod "6b7e3053-63a4-43c2-ada7-f98132ef6d4d" (UID: "6b7e3053-63a4-43c2-ada7-f98132ef6d4d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.761834 5385 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-credential-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.761855 5385 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-workload-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.761872 5385 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-data\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.761884 5385 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istiod-ca-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.761896 5385 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-workload-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.762568 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.761909 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-token" pod "6b7e3053-63a4-43c2-ada7-f98132ef6d4d" (UID: "6b7e3053-63a4-43c2-ada7-f98132ef6d4d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:39:44.768176 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.764442 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-podinfo" pod "6b7e3053-63a4-43c2-ada7-f98132ef6d4d" (UID: "6b7e3053-63a4-43c2-ada7-f98132ef6d4d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 07:39:44.768176 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.765930 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-kube-api-access-8xwt2" pod "6b7e3053-63a4-43c2-ada7-f98132ef6d4d" (UID: "6b7e3053-63a4-43c2-ada7-f98132ef6d4d"). InnerVolumeSpecName "kube-api-access-8xwt2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:39:44.768176 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.768143 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-envoy" pod "6b7e3053-63a4-43c2-ada7-f98132ef6d4d" (UID: "6b7e3053-63a4-43c2-ada7-f98132ef6d4d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:44.862885 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.862857 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-data\") pod \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " Nov 04 07:39:44.863054 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.862897 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-istio-envoy\") pod \"cb02a450-1430-46b8-95bd-94edf3e27328\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " Nov 04 07:39:44.863054 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.862944 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-kube-api-access-gbpg2\" (UniqueName: \"kubernetes.io/projected/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-kube-api-access-gbpg2\") pod \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " Nov 04 07:39:44.863054 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.862976 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-istio-data\") pod \"cb02a450-1430-46b8-95bd-94edf3e27328\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " Nov 04 07:39:44.863054 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863004 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-credential-socket\") pod \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " Nov 04 07:39:44.863054 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863030 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-token\" (UniqueName: \"kubernetes.io/projected/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-token\") pod \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " Nov 04 07:39:44.863331 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863095 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istiod-ca-cert\") pod \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " Nov 04 07:39:44.863331 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863132 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-workload-socket\") pod \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " Nov 04 07:39:44.863331 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863207 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb02a450-1430-46b8-95bd-94edf3e27328-istio-token\" (UniqueName: \"kubernetes.io/projected/cb02a450-1430-46b8-95bd-94edf3e27328-istio-token\") pod \"cb02a450-1430-46b8-95bd-94edf3e27328\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " Nov 04 07:39:44.863331 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863200 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-data" pod "e977f4f4-c3aa-484d-b64a-fda4d19ce7de" (UID: "e977f4f4-c3aa-484d-b64a-fda4d19ce7de"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:44.863331 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863296 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-podinfo\") pod \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " Nov 04 07:39:44.863331 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863299 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-istio-data" pod "cb02a450-1430-46b8-95bd-94edf3e27328" (UID: "cb02a450-1430-46b8-95bd-94edf3e27328"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:44.863331 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863313 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-credential-socket" pod "e977f4f4-c3aa-484d-b64a-fda4d19ce7de" (UID: "e977f4f4-c3aa-484d-b64a-fda4d19ce7de"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:44.863794 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863361 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-credential-socket\") pod \"cb02a450-1430-46b8-95bd-94edf3e27328\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " Nov 04 07:39:44.863794 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863391 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cb02a450-1430-46b8-95bd-94edf3e27328-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb02a450-1430-46b8-95bd-94edf3e27328-istio-podinfo\") pod \"cb02a450-1430-46b8-95bd-94edf3e27328\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " Nov 04 07:39:44.863794 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863451 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-workload-certs\") pod \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " Nov 04 07:39:44.863794 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863469 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-workload-socket" pod "e977f4f4-c3aa-484d-b64a-fda4d19ce7de" (UID: "e977f4f4-c3aa-484d-b64a-fda4d19ce7de"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:44.863794 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863481 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-workload-socket\") pod \"cb02a450-1430-46b8-95bd-94edf3e27328\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " Nov 04 07:39:44.863794 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863552 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cb02a450-1430-46b8-95bd-94edf3e27328-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb02a450-1430-46b8-95bd-94edf3e27328-istiod-ca-cert\") pod \"cb02a450-1430-46b8-95bd-94edf3e27328\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " Nov 04 07:39:44.863794 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863586 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istiod-ca-cert" pod "e977f4f4-c3aa-484d-b64a-fda4d19ce7de" (UID: "e977f4f4-c3aa-484d-b64a-fda4d19ce7de"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:39:44.863794 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863591 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb02a450-1430-46b8-95bd-94edf3e27328-kube-api-access-74vbq\" (UniqueName: \"kubernetes.io/projected/cb02a450-1430-46b8-95bd-94edf3e27328-kube-api-access-74vbq\") pod \"cb02a450-1430-46b8-95bd-94edf3e27328\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " Nov 04 07:39:44.863794 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863690 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-workload-socket" pod "cb02a450-1430-46b8-95bd-94edf3e27328" (UID: "cb02a450-1430-46b8-95bd-94edf3e27328"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:44.863794 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863731 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-envoy\") pod \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\" (UID: \"e977f4f4-c3aa-484d-b64a-fda4d19ce7de\") " Nov 04 07:39:44.865608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863843 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-workload-certs\") pod \"cb02a450-1430-46b8-95bd-94edf3e27328\" (UID: \"cb02a450-1430-46b8-95bd-94edf3e27328\") " Nov 04 07:39:44.865608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.863992 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-workload-certs" pod "e977f4f4-c3aa-484d-b64a-fda4d19ce7de" (UID: "e977f4f4-c3aa-484d-b64a-fda4d19ce7de"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:44.865608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.864439 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-credential-socket" pod "cb02a450-1430-46b8-95bd-94edf3e27328" (UID: "cb02a450-1430-46b8-95bd-94edf3e27328"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:44.865608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.864721 5385 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-podinfo\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.865608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.864729 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-workload-certs" pod "cb02a450-1430-46b8-95bd-94edf3e27328" (UID: "cb02a450-1430-46b8-95bd-94edf3e27328"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:44.865608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.864746 5385 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-data\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.865608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.864789 5385 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-istio-data\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.865608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.864807 5385 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-credential-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.865608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.864822 5385 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istiod-ca-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.865608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.864837 5385 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-workload-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.865608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.864876 5385 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-token\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.865608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.864893 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xwt2\" (UniqueName: \"kubernetes.io/projected/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-kube-api-access-8xwt2\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.865608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.864907 5385 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b7e3053-63a4-43c2-ada7-f98132ef6d4d-istio-envoy\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.865608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.864950 5385 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-credential-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.865608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.864968 5385 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-workload-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.865608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.864982 5385 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-workload-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.865608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.865066 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb02a450-1430-46b8-95bd-94edf3e27328-istiod-ca-cert" pod "cb02a450-1430-46b8-95bd-94edf3e27328" (UID: "cb02a450-1430-46b8-95bd-94edf3e27328"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:39:44.866449 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.866323 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-kube-api-access-gbpg2" pod "e977f4f4-c3aa-484d-b64a-fda4d19ce7de" (UID: "e977f4f4-c3aa-484d-b64a-fda4d19ce7de"). InnerVolumeSpecName "kube-api-access-gbpg2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:39:44.867256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.867229 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-podinfo" pod "e977f4f4-c3aa-484d-b64a-fda4d19ce7de" (UID: "e977f4f4-c3aa-484d-b64a-fda4d19ce7de"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 07:39:44.868157 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.868130 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-envoy" pod "e977f4f4-c3aa-484d-b64a-fda4d19ce7de" (UID: "e977f4f4-c3aa-484d-b64a-fda4d19ce7de"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:44.868157 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.868147 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb02a450-1430-46b8-95bd-94edf3e27328-kube-api-access-74vbq" pod "cb02a450-1430-46b8-95bd-94edf3e27328" (UID: "cb02a450-1430-46b8-95bd-94edf3e27328"). InnerVolumeSpecName "kube-api-access-74vbq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:39:44.868381 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.868150 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb02a450-1430-46b8-95bd-94edf3e27328-istio-token" pod "cb02a450-1430-46b8-95bd-94edf3e27328" (UID: "cb02a450-1430-46b8-95bd-94edf3e27328"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:39:44.868381 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.868146 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cb02a450-1430-46b8-95bd-94edf3e27328-istio-podinfo" pod "cb02a450-1430-46b8-95bd-94edf3e27328" (UID: "cb02a450-1430-46b8-95bd-94edf3e27328"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 07:39:44.868381 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.868234 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-istio-envoy" pod "cb02a450-1430-46b8-95bd-94edf3e27328" (UID: "cb02a450-1430-46b8-95bd-94edf3e27328"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:39:44.869418 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.869393 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-token" pod "e977f4f4-c3aa-484d-b64a-fda4d19ce7de" (UID: "e977f4f4-c3aa-484d-b64a-fda4d19ce7de"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:39:44.966710 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.966679 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbpg2\" (UniqueName: \"kubernetes.io/projected/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-kube-api-access-gbpg2\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.966710 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.966706 5385 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-token\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.966911 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.966722 5385 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb02a450-1430-46b8-95bd-94edf3e27328-istio-token\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.966911 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.966738 5385 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-podinfo\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.966911 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.966753 5385 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb02a450-1430-46b8-95bd-94edf3e27328-istio-podinfo\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.966911 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.966765 5385 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb02a450-1430-46b8-95bd-94edf3e27328-istiod-ca-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.966911 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.966778 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74vbq\" (UniqueName: \"kubernetes.io/projected/cb02a450-1430-46b8-95bd-94edf3e27328-kube-api-access-74vbq\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.966911 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.966791 5385 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e977f4f4-c3aa-484d-b64a-fda4d19ce7de-istio-envoy\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.966911 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.966804 5385 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-workload-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:44.966911 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:44.966819 5385 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb02a450-1430-46b8-95bd-94edf3e27328-istio-envoy\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:39:45.296291 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:45.296250 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:45.569007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:45.568929 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" Nov 04 07:39:45.569164 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:45.568928 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp" event={"ID":"cb02a450-1430-46b8-95bd-94edf3e27328","Type":"ContainerDied","Data":"068efed73ad911d0b8376399752358453a874078a69aec10649a07d9064388cc"} Nov 04 07:39:45.569164 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:45.569087 5385 scope.go:122] "RemoveContainer" containerID="34e23a4b4f5503b92d5c7289b953e6619a4c133323ebeb62f94b23a9788edc7a" Nov 04 07:39:45.571120 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:45.570792 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9" Nov 04 07:39:45.572726 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:45.572682 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" Nov 04 07:39:45.572976 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:45.572700 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj" event={"ID":"e977f4f4-c3aa-484d-b64a-fda4d19ce7de","Type":"ContainerDied","Data":"d8034395390716756480fd4aad5bf753b4370e177619c104703bc867399fb049"} Nov 04 07:39:45.582981 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:45.582955 5385 scope.go:122] "RemoveContainer" containerID="7904b10e7609f2deaa622d030e6cd8627ff930057896fe0e905d8ebdd8ec0c31" Nov 04 07:39:45.617271 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:45.617247 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9"] Nov 04 07:39:45.623082 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:45.623061 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-4q7l9"] Nov 04 07:39:45.636801 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:45.636775 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj"] Nov 04 07:39:45.646395 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:45.646362 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-945qj"] Nov 04 07:39:45.659962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:45.659941 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp"] Nov 04 07:39:45.664342 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:45.664321 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-60ed4a-openshift-default-76f5998f8c-tljvp"] Nov 04 07:39:46.157184 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:46.157146 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:46.813469 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:46.813428 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6b7e3053-63a4-43c2-ada7-f98132ef6d4d" path="/var/lib/kubelet/pods/6b7e3053-63a4-43c2-ada7-f98132ef6d4d/volumes" Nov 04 07:39:46.813960 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:46.813927 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb02a450-1430-46b8-95bd-94edf3e27328" path="/var/lib/kubelet/pods/cb02a450-1430-46b8-95bd-94edf3e27328/volumes" Nov 04 07:39:46.814342 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:46.814324 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e977f4f4-c3aa-484d-b64a-fda4d19ce7de" path="/var/lib/kubelet/pods/e977f4f4-c3aa-484d-b64a-fda4d19ce7de/volumes" Nov 04 07:39:46.979483 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:46.979442 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:47.813169 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:47.813128 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:48.015990 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:48.015943 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:48.597075 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:48.597045 5385 generic.go:356] "Generic (PLEG): container finished" podID="63ab0edc-d9ba-486d-bfc0-7060678daea2" containerID="19a36bedda2349d3371d0983319133ed846dd32d1e66982ff1cb0c7af13512f2" exitCode=0 Nov 04 07:39:48.597270 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:48.597118 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" event={"ID":"63ab0edc-d9ba-486d-bfc0-7060678daea2","Type":"ContainerDied","Data":"19a36bedda2349d3371d0983319133ed846dd32d1e66982ff1cb0c7af13512f2"} Nov 04 07:39:48.597559 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:48.597545 5385 scope.go:122] "RemoveContainer" containerID="19a36bedda2349d3371d0983319133ed846dd32d1e66982ff1cb0c7af13512f2" Nov 04 07:39:48.599106 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:48.599081 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:39:48.599185 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:48.599158 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:39:48.599225 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:48.599189 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:39:48.983013 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:48.982968 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:49.377784 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:49.377723 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:49.569530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:49.569353 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:49.605614 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:49.605577 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" event={"ID":"63ab0edc-d9ba-486d-bfc0-7060678daea2","Type":"ContainerStarted","Data":"22dc93cc55bf4ca7de359d7177577e9d96ebcf20e0ee389496a80836719add1c"} Nov 04 07:39:49.679468 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:49.679393 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:49.987825 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:49.987744 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:50.138371 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:50.138334 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:50.199237 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:50.199208 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:50.199444 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:50.199424 5385 patch_prober.go:29] interesting pod/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Nov 04 07:39:50.199494 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:50.199469 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" podUID="63ab0edc-d9ba-486d-bfc0-7060678daea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Nov 04 07:39:50.296578 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:50.290652 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:50.874885 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:50.874839 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:51.014657 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:51.014621 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:51.199131 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:51.199031 5385 patch_prober.go:29] interesting pod/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Nov 04 07:39:51.199131 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:51.199091 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" podUID="63ab0edc-d9ba-486d-bfc0-7060678daea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Nov 04 07:39:51.431646 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:51.431606 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:51.668994 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:51.668937 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:52.199324 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:52.199290 5385 patch_prober.go:29] interesting pod/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Nov 04 07:39:52.199746 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:52.199355 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" podUID="63ab0edc-d9ba-486d-bfc0-7060678daea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Nov 04 07:39:52.488969 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:52.488873 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:53.099940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:53.099883 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:53.207140 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:53.207088 5385 patch_prober.go:29] interesting pod/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Nov 04 07:39:53.207792 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:53.207756 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" podUID="63ab0edc-d9ba-486d-bfc0-7060678daea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Nov 04 07:39:53.224203 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:53.224156 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:53.337660 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:53.337612 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:53.440245 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:53.440198 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:54.200582 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:54.200105 5385 patch_prober.go:29] interesting pod/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Nov 04 07:39:54.200582 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:54.200178 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" podUID="63ab0edc-d9ba-486d-bfc0-7060678daea2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Nov 04 07:39:54.460093 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:54.459982 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:55.226326 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:55.226292 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:55.226747 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:55.226722 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:55.229074 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:55.229052 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:39:55.306214 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:55.306174 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:56.090786 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:56.090753 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:57.176905 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:57.176872 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:57.987343 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:57.987254 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:58.876147 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:58.876113 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:59.675434 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:59.675385 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:39:59.986355 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:39:59.986261 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:00.086276 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:00.086239 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:00.451238 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:00.451208 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:00.570313 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:00.570272 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:00.875483 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:40:00.875451 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:40:01.283716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:01.283633 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:02.083099 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:02.082996 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:02.884256 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:02.884208 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:03.670891 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:03.670855 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:04.478802 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:04.478766 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:05.315018 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:05.314983 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:05.761632 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:05.761599 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:05.872212 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:05.872170 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:06.178030 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:06.177994 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:07.347317 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:07.347280 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:07.532817 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:07.532778 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:07.638681 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:07.638642 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww"] Nov 04 07:40:07.843004 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:07.842973 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:07.978097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:07.978002 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:08.173490 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:08.173452 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:08.441327 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:08.439568 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:08.545535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:08.545478 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:08.680498 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:08.680460 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:08.857912 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:08.857836 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:09.001652 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:09.001617 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh"] Nov 04 07:40:09.101738 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:09.101709 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:09.206517 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:09.206477 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:09.691367 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:09.691335 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:10.670530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:10.670472 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:11.577929 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:11.577893 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:12.370118 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:12.370078 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:12.639683 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:12.639575 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" podUID="808ce2ba-76f7-4825-986d-4d1030e4737e" containerName="payload-processing" containerID="cri-o://e08df657e4ceab34f1c5cb5d884d86efb92a4865aab2166e5aaf48a6230dca5f" gracePeriod=25 Nov 04 07:40:12.855159 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:12.855126 5385 generic.go:356] "Generic (PLEG): container finished" podID="808ce2ba-76f7-4825-986d-4d1030e4737e" containerID="e08df657e4ceab34f1c5cb5d884d86efb92a4865aab2166e5aaf48a6230dca5f" exitCode=0 Nov 04 07:40:12.855351 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:12.855205 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" event={"ID":"808ce2ba-76f7-4825-986d-4d1030e4737e","Type":"ContainerDied","Data":"e08df657e4ceab34f1c5cb5d884d86efb92a4865aab2166e5aaf48a6230dca5f"} Nov 04 07:40:12.920969 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:12.920939 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:40:13.015316 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.015284 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/808ce2ba-76f7-4825-986d-4d1030e4737e-kube-api-access-hsbbm\" (UniqueName: \"kubernetes.io/projected/808ce2ba-76f7-4825-986d-4d1030e4737e-kube-api-access-hsbbm\") pod \"808ce2ba-76f7-4825-986d-4d1030e4737e\" (UID: \"808ce2ba-76f7-4825-986d-4d1030e4737e\") " Nov 04 07:40:13.016720 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.015378 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/808ce2ba-76f7-4825-986d-4d1030e4737e-provider-credentials\" (UniqueName: \"kubernetes.io/projected/808ce2ba-76f7-4825-986d-4d1030e4737e-provider-credentials\") pod \"808ce2ba-76f7-4825-986d-4d1030e4737e\" (UID: \"808ce2ba-76f7-4825-986d-4d1030e4737e\") " Nov 04 07:40:13.016720 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.015470 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/808ce2ba-76f7-4825-986d-4d1030e4737e-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/808ce2ba-76f7-4825-986d-4d1030e4737e-routing-overlay-volume\") pod \"808ce2ba-76f7-4825-986d-4d1030e4737e\" (UID: \"808ce2ba-76f7-4825-986d-4d1030e4737e\") " Nov 04 07:40:13.016720 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.015658 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/808ce2ba-76f7-4825-986d-4d1030e4737e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/808ce2ba-76f7-4825-986d-4d1030e4737e-plugins-config-volume\") pod \"808ce2ba-76f7-4825-986d-4d1030e4737e\" (UID: \"808ce2ba-76f7-4825-986d-4d1030e4737e\") " Nov 04 07:40:13.016720 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.015928 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/808ce2ba-76f7-4825-986d-4d1030e4737e-routing-overlay-volume" pod "808ce2ba-76f7-4825-986d-4d1030e4737e" (UID: "808ce2ba-76f7-4825-986d-4d1030e4737e"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:40:13.016720 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.016020 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/808ce2ba-76f7-4825-986d-4d1030e4737e-plugins-config-volume" pod "808ce2ba-76f7-4825-986d-4d1030e4737e" (UID: "808ce2ba-76f7-4825-986d-4d1030e4737e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:40:13.016720 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.016380 5385 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/808ce2ba-76f7-4825-986d-4d1030e4737e-routing-overlay-volume\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:40:13.016720 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.016396 5385 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/808ce2ba-76f7-4825-986d-4d1030e4737e-plugins-config-volume\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:40:13.018113 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.018088 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/808ce2ba-76f7-4825-986d-4d1030e4737e-kube-api-access-hsbbm" pod "808ce2ba-76f7-4825-986d-4d1030e4737e" (UID: "808ce2ba-76f7-4825-986d-4d1030e4737e"). InnerVolumeSpecName "kube-api-access-hsbbm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:40:13.018214 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.018196 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/808ce2ba-76f7-4825-986d-4d1030e4737e-provider-credentials" pod "808ce2ba-76f7-4825-986d-4d1030e4737e" (UID: "808ce2ba-76f7-4825-986d-4d1030e4737e"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:40:13.118152 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.118116 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsbbm\" (UniqueName: \"kubernetes.io/projected/808ce2ba-76f7-4825-986d-4d1030e4737e-kube-api-access-hsbbm\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:40:13.118152 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.118146 5385 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/808ce2ba-76f7-4825-986d-4d1030e4737e-provider-credentials\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:40:13.256582 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.241351 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:13.863860 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.863824 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" Nov 04 07:40:13.864314 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.863826 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww" event={"ID":"808ce2ba-76f7-4825-986d-4d1030e4737e","Type":"ContainerDied","Data":"0a6ffa64c9b026c88cbe3ab9a680b77507bc16aa5735a41b7c22d31aa171c936"} Nov 04 07:40:13.864314 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.863968 5385 scope.go:122] "RemoveContainer" containerID="e08df657e4ceab34f1c5cb5d884d86efb92a4865aab2166e5aaf48a6230dca5f" Nov 04 07:40:13.891821 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.891786 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww"] Nov 04 07:40:13.896474 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.896444 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-023307/payload-processing-external-model-e2e-extmod-b-023307-5bcb44lww"] Nov 04 07:40:13.976418 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:13.976379 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:14.002019 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.001956 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" podUID="b3a8327a-27f1-4947-8567-49372cc916ec" containerName="payload-pre-processing" containerID="cri-o://4b57ad09d8630f4510526b60e1e3c01a9391a9347dfe9de7335831b55d9072e3" gracePeriod=25 Nov 04 07:40:14.261584 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.261554 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" Nov 04 07:40:14.346345 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.346314 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b3a8327a-27f1-4947-8567-49372cc916ec-kube-api-access-7946m\" (UniqueName: \"kubernetes.io/projected/b3a8327a-27f1-4947-8567-49372cc916ec-kube-api-access-7946m\") pod \"b3a8327a-27f1-4947-8567-49372cc916ec\" (UID: \"b3a8327a-27f1-4947-8567-49372cc916ec\") " Nov 04 07:40:14.346563 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.346358 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b3a8327a-27f1-4947-8567-49372cc916ec-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3a8327a-27f1-4947-8567-49372cc916ec-plugins-config-volume\") pod \"b3a8327a-27f1-4947-8567-49372cc916ec\" (UID: \"b3a8327a-27f1-4947-8567-49372cc916ec\") " Nov 04 07:40:14.346815 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.346780 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3a8327a-27f1-4947-8567-49372cc916ec-plugins-config-volume" pod "b3a8327a-27f1-4947-8567-49372cc916ec" (UID: "b3a8327a-27f1-4947-8567-49372cc916ec"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:40:14.346941 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.346928 5385 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3a8327a-27f1-4947-8567-49372cc916ec-plugins-config-volume\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:40:14.348538 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.348517 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3a8327a-27f1-4947-8567-49372cc916ec-kube-api-access-7946m" pod "b3a8327a-27f1-4947-8567-49372cc916ec" (UID: "b3a8327a-27f1-4947-8567-49372cc916ec"). InnerVolumeSpecName "kube-api-access-7946m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:40:14.448008 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.447922 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7946m\" (UniqueName: \"kubernetes.io/projected/b3a8327a-27f1-4947-8567-49372cc916ec-kube-api-access-7946m\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:40:14.763891 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.763799 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:14.835420 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.835372 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="808ce2ba-76f7-4825-986d-4d1030e4737e" path="/var/lib/kubelet/pods/808ce2ba-76f7-4825-986d-4d1030e4737e/volumes" Nov 04 07:40:14.883136 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.883077 5385 generic.go:356] "Generic (PLEG): container finished" podID="b3a8327a-27f1-4947-8567-49372cc916ec" containerID="4b57ad09d8630f4510526b60e1e3c01a9391a9347dfe9de7335831b55d9072e3" exitCode=0 Nov 04 07:40:14.883768 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.883174 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" Nov 04 07:40:14.883915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.883197 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" event={"ID":"b3a8327a-27f1-4947-8567-49372cc916ec","Type":"ContainerDied","Data":"4b57ad09d8630f4510526b60e1e3c01a9391a9347dfe9de7335831b55d9072e3"} Nov 04 07:40:14.883987 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.883957 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh" event={"ID":"b3a8327a-27f1-4947-8567-49372cc916ec","Type":"ContainerDied","Data":"e24126d69fc216ebec367ac2e10631fbf462afc67d4d060d92f18cc3bdfdb56f"} Nov 04 07:40:14.884039 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.883997 5385 scope.go:122] "RemoveContainer" containerID="4b57ad09d8630f4510526b60e1e3c01a9391a9347dfe9de7335831b55d9072e3" Nov 04 07:40:14.895617 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.895486 5385 scope.go:122] "RemoveContainer" containerID="4b57ad09d8630f4510526b60e1e3c01a9391a9347dfe9de7335831b55d9072e3" Nov 04 07:40:14.895872 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:40:14.895847 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b57ad09d8630f4510526b60e1e3c01a9391a9347dfe9de7335831b55d9072e3\": container with ID starting with 4b57ad09d8630f4510526b60e1e3c01a9391a9347dfe9de7335831b55d9072e3 not found: ID does not exist" containerID="4b57ad09d8630f4510526b60e1e3c01a9391a9347dfe9de7335831b55d9072e3" Nov 04 07:40:14.895941 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.895881 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b57ad09d8630f4510526b60e1e3c01a9391a9347dfe9de7335831b55d9072e3"} err="failed to get container status \"4b57ad09d8630f4510526b60e1e3c01a9391a9347dfe9de7335831b55d9072e3\": rpc error: code = NotFound desc = could not find container \"4b57ad09d8630f4510526b60e1e3c01a9391a9347dfe9de7335831b55d9072e3\": container with ID starting with 4b57ad09d8630f4510526b60e1e3c01a9391a9347dfe9de7335831b55d9072e3 not found: ID does not exist" Nov 04 07:40:14.908668 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.908637 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh"] Nov 04 07:40:14.913530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:14.913485 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-6dd194-56669f7d55-6nmjh"] Nov 04 07:40:15.384096 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:15.384057 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:15.487238 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:15.487204 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:15.621641 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:15.621596 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:15.836417 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:15.836331 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:16.454100 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:16.454050 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:16.643527 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:16.639014 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:16.813169 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:16.813091 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b3a8327a-27f1-4947-8567-49372cc916ec" path="/var/lib/kubelet/pods/b3a8327a-27f1-4947-8567-49372cc916ec/volumes" Nov 04 07:40:17.497839 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:17.497809 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:18.243630 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:18.243596 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x"] Nov 04 07:40:18.243836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:18.243811 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" podUID="f6e3296f-cbf9-43d0-8a3f-86e6d0190b63" containerName="maas-api" containerID="cri-o://9273cdd09d306a9bf1d580fdd8c00f8273fc318006fc8e5404d7e94e54b18111" gracePeriod=30 Nov 04 07:40:18.294577 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:18.294549 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:18.685916 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:18.685890 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:40:18.836901 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:18.836872 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-openshift-service-ca\") pod \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\" (UID: \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\") " Nov 04 07:40:18.837099 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:18.836969 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-maas-api-tls\") pod \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\" (UID: \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\") " Nov 04 07:40:18.837099 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:18.836991 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-kube-api-access-s67wh\" (UniqueName: \"kubernetes.io/projected/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-kube-api-access-s67wh\") pod \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\" (UID: \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\") " Nov 04 07:40:18.837099 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:18.837073 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-metrics-certs\") pod \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\" (UID: \"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63\") " Nov 04 07:40:18.837292 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:18.837224 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-openshift-service-ca" pod "f6e3296f-cbf9-43d0-8a3f-86e6d0190b63" (UID: "f6e3296f-cbf9-43d0-8a3f-86e6d0190b63"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:40:18.837619 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:18.837588 5385 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-openshift-service-ca\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:40:18.839248 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:18.839222 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-kube-api-access-s67wh" pod "f6e3296f-cbf9-43d0-8a3f-86e6d0190b63" (UID: "f6e3296f-cbf9-43d0-8a3f-86e6d0190b63"). InnerVolumeSpecName "kube-api-access-s67wh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:40:18.839360 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:18.839331 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-metrics-certs" pod "f6e3296f-cbf9-43d0-8a3f-86e6d0190b63" (UID: "f6e3296f-cbf9-43d0-8a3f-86e6d0190b63"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:40:18.839435 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:18.839419 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-maas-api-tls" pod "f6e3296f-cbf9-43d0-8a3f-86e6d0190b63" (UID: "f6e3296f-cbf9-43d0-8a3f-86e6d0190b63"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:40:18.980602 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:18.977195 5385 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-maas-api-tls\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:40:18.980602 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:18.977363 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s67wh\" (UniqueName: \"kubernetes.io/projected/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-kube-api-access-s67wh\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:40:18.980602 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:18.977384 5385 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63-metrics-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:40:19.005015 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:19.004973 5385 generic.go:356] "Generic (PLEG): container finished" podID="f6e3296f-cbf9-43d0-8a3f-86e6d0190b63" containerID="9273cdd09d306a9bf1d580fdd8c00f8273fc318006fc8e5404d7e94e54b18111" exitCode=0 Nov 04 07:40:19.005378 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:19.005171 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" Nov 04 07:40:19.006558 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:19.005181 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" event={"ID":"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63","Type":"ContainerDied","Data":"9273cdd09d306a9bf1d580fdd8c00f8273fc318006fc8e5404d7e94e54b18111"} Nov 04 07:40:19.006558 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:19.005757 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x" event={"ID":"f6e3296f-cbf9-43d0-8a3f-86e6d0190b63","Type":"ContainerDied","Data":"43e7b850a33ecb75f15406b8de4f6a84507454821427b9cc2102bc2258a7480a"} Nov 04 07:40:19.006558 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:19.005833 5385 scope.go:122] "RemoveContainer" containerID="9273cdd09d306a9bf1d580fdd8c00f8273fc318006fc8e5404d7e94e54b18111" Nov 04 07:40:19.030758 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:19.030732 5385 scope.go:122] "RemoveContainer" containerID="9273cdd09d306a9bf1d580fdd8c00f8273fc318006fc8e5404d7e94e54b18111" Nov 04 07:40:19.031132 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:40:19.031099 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9273cdd09d306a9bf1d580fdd8c00f8273fc318006fc8e5404d7e94e54b18111\": container with ID starting with 9273cdd09d306a9bf1d580fdd8c00f8273fc318006fc8e5404d7e94e54b18111 not found: ID does not exist" containerID="9273cdd09d306a9bf1d580fdd8c00f8273fc318006fc8e5404d7e94e54b18111" Nov 04 07:40:19.031218 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:19.031145 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9273cdd09d306a9bf1d580fdd8c00f8273fc318006fc8e5404d7e94e54b18111"} err="failed to get container status \"9273cdd09d306a9bf1d580fdd8c00f8273fc318006fc8e5404d7e94e54b18111\": rpc error: code = NotFound desc = could not find container \"9273cdd09d306a9bf1d580fdd8c00f8273fc318006fc8e5404d7e94e54b18111\": container with ID starting with 9273cdd09d306a9bf1d580fdd8c00f8273fc318006fc8e5404d7e94e54b18111 not found: ID does not exist" Nov 04 07:40:19.040798 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:19.040773 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x"] Nov 04 07:40:19.048977 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:19.048950 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-6dd194-7b79f75b68-m5d2x"] Nov 04 07:40:19.147498 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:19.147463 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:19.672408 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:19.672361 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:20.480928 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:20.480887 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:20.813704 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:20.813611 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6e3296f-cbf9-43d0-8a3f-86e6d0190b63" path="/var/lib/kubelet/pods/f6e3296f-cbf9-43d0-8a3f-86e6d0190b63/volumes" Nov 04 07:40:21.401857 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:21.401820 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:22.180703 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:22.180670 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:22.815009 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:22.814979 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:22.934538 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:22.934328 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:23.161834 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:23.161799 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:23.531216 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:23.531117 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:23.646384 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:23.646342 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:23.764367 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:23.764327 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:23.911588 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:23.909329 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:24.017115 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:24.017080 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:24.407412 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:24.407382 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:24.521226 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:24.521194 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:24.660616 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:24.660499 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:25.165908 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:25.165868 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:26.086287 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:26.086252 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:26.983264 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:26.983230 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:27.969474 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:27.969434 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:28.792028 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:28.791992 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:28.953255 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:28.953216 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:29.811329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:29.811270 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:30.589434 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:30.589399 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:30.876562 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:40:30.876527 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:40:31.389614 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:31.389582 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:31.657290 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:31.657205 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5"] Nov 04 07:40:31.732801 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:31.732774 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:31.833702 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:31.833667 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:32.235708 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:32.235675 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:32.476938 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:32.476904 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:33.279997 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:33.279964 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:34.082382 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:34.082342 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:34.878545 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:34.878496 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:35.688965 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:35.688921 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:36.657843 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:36.657792 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" podUID="947a11da-cef0-4a9c-8e51-568588b5885a" containerName="payload-processing" containerID="cri-o://26baebbe1d2e92d97dca787d9fa3f88f8e8c3cbd6063a6daf985f224bb51177b" gracePeriod=25 Nov 04 07:40:36.885953 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:36.885864 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:37.039663 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:37.039388 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" Nov 04 07:40:37.108102 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:37.108069 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/947a11da-cef0-4a9c-8e51-568588b5885a-kube-api-access-6fnzt\" (UniqueName: \"kubernetes.io/projected/947a11da-cef0-4a9c-8e51-568588b5885a-kube-api-access-6fnzt\") pod \"947a11da-cef0-4a9c-8e51-568588b5885a\" (UID: \"947a11da-cef0-4a9c-8e51-568588b5885a\") " Nov 04 07:40:37.108290 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:37.108228 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/947a11da-cef0-4a9c-8e51-568588b5885a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/947a11da-cef0-4a9c-8e51-568588b5885a-plugins-config-volume\") pod \"947a11da-cef0-4a9c-8e51-568588b5885a\" (UID: \"947a11da-cef0-4a9c-8e51-568588b5885a\") " Nov 04 07:40:37.109003 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:37.108978 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/947a11da-cef0-4a9c-8e51-568588b5885a-plugins-config-volume" pod "947a11da-cef0-4a9c-8e51-568588b5885a" (UID: "947a11da-cef0-4a9c-8e51-568588b5885a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:40:37.111547 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:37.111521 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/947a11da-cef0-4a9c-8e51-568588b5885a-kube-api-access-6fnzt" pod "947a11da-cef0-4a9c-8e51-568588b5885a" (UID: "947a11da-cef0-4a9c-8e51-568588b5885a"). InnerVolumeSpecName "kube-api-access-6fnzt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:40:37.207596 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:37.207064 5385 generic.go:356] "Generic (PLEG): container finished" podID="947a11da-cef0-4a9c-8e51-568588b5885a" containerID="26baebbe1d2e92d97dca787d9fa3f88f8e8c3cbd6063a6daf985f224bb51177b" exitCode=0 Nov 04 07:40:37.207596 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:37.207191 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" event={"ID":"947a11da-cef0-4a9c-8e51-568588b5885a","Type":"ContainerDied","Data":"26baebbe1d2e92d97dca787d9fa3f88f8e8c3cbd6063a6daf985f224bb51177b"} Nov 04 07:40:37.207596 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:37.207246 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" event={"ID":"947a11da-cef0-4a9c-8e51-568588b5885a","Type":"ContainerDied","Data":"6e6b8f23bad13cf33e3ab13b6bb0fcacca30e1d922c4aa9cd59facb98b3e9a3c"} Nov 04 07:40:37.207596 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:37.207279 5385 scope.go:122] "RemoveContainer" containerID="26baebbe1d2e92d97dca787d9fa3f88f8e8c3cbd6063a6daf985f224bb51177b" Nov 04 07:40:37.207596 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:37.207438 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5" Nov 04 07:40:37.214757 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:37.214713 5385 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/947a11da-cef0-4a9c-8e51-568588b5885a-plugins-config-volume\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:40:37.214757 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:37.214749 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fnzt\" (UniqueName: \"kubernetes.io/projected/947a11da-cef0-4a9c-8e51-568588b5885a-kube-api-access-6fnzt\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:40:37.233329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:37.233260 5385 scope.go:122] "RemoveContainer" containerID="26baebbe1d2e92d97dca787d9fa3f88f8e8c3cbd6063a6daf985f224bb51177b" Nov 04 07:40:37.233801 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:40:37.233770 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26baebbe1d2e92d97dca787d9fa3f88f8e8c3cbd6063a6daf985f224bb51177b\": container with ID starting with 26baebbe1d2e92d97dca787d9fa3f88f8e8c3cbd6063a6daf985f224bb51177b not found: ID does not exist" containerID="26baebbe1d2e92d97dca787d9fa3f88f8e8c3cbd6063a6daf985f224bb51177b" Nov 04 07:40:37.233914 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:37.233813 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26baebbe1d2e92d97dca787d9fa3f88f8e8c3cbd6063a6daf985f224bb51177b"} err="failed to get container status \"26baebbe1d2e92d97dca787d9fa3f88f8e8c3cbd6063a6daf985f224bb51177b\": rpc error: code = NotFound desc = could not find container \"26baebbe1d2e92d97dca787d9fa3f88f8e8c3cbd6063a6daf985f224bb51177b\": container with ID starting with 26baebbe1d2e92d97dca787d9fa3f88f8e8c3cbd6063a6daf985f224bb51177b not found: ID does not exist" Nov 04 07:40:37.321594 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:37.321544 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5"] Nov 04 07:40:37.326783 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:37.326752 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-812032-99cfb6886-75dl5"] Nov 04 07:40:38.229589 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:38.229551 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:38.830670 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:38.830621 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="947a11da-cef0-4a9c-8e51-568588b5885a" path="/var/lib/kubelet/pods/947a11da-cef0-4a9c-8e51-568588b5885a/volumes" Nov 04 07:40:39.133414 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:39.133375 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:39.684711 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:39.684676 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:40.590708 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:40.590674 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:41.401769 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:41.401734 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:41.810916 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:41.810824 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:42.581974 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:42.581937 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:43.383147 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:43.383097 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:44.191606 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:44.191569 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:45.186741 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:45.186702 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:45.804070 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:45.804034 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:45.902917 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:45.902865 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:46.087076 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:46.086985 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:46.381226 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:46.381185 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:46.734686 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:46.734599 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:47.199143 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:47.199078 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:47.350060 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:47.350018 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:48.394925 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:48.394887 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:49.581013 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:49.580978 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:50.477280 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:50.477246 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:51.503531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:51.503471 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:51.924178 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:51.924140 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:52.034124 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:52.034094 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:52.273582 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:52.264582 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks"] Nov 04 07:40:52.379698 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:52.379668 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:52.574557 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:52.574471 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:52.704525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:52.694039 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:52.959135 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:52.959069 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:53.583344 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:53.583309 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:54.576659 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:54.576625 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:55.545130 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:55.545091 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:56.304777 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:56.304742 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:57.073753 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:57.073721 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:57.889686 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:57.889643 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:58.774809 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:58.774773 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:58.874497 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:58.874457 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:40:59.267336 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:40:59.267197 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:00.178971 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:00.178935 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:00.312701 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:00.312658 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:00.669736 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:00.669698 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:00.875298 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:41:00.875263 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:41:01.457197 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:01.457159 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:01.613046 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:01.613013 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:01.755356 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:01.755259 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:02.016970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:02.016885 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:02.226052 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:02.226012 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:02.673799 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:02.673769 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:03.784284 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:03.784249 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:04.808655 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:04.791478 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:05.723786 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:05.723701 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:06.497356 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:06.497321 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:07.285882 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.285834 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:07.337528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.332271 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" podUID="128cb6a9-354b-4beb-9ea7-eac997fc81b9" containerName="main" containerID="cri-o://74dc4f1cc0dcabb75ebf98701f33acbbda5ef4ae35688d7b4ab589fad0f91962" gracePeriod=45 Nov 04 07:41:07.431673 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.431628 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:07.431945 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.431910 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" podUID="128cb6a9-354b-4beb-9ea7-eac997fc81b9" containerName="main" probeResult="failure" output="Get \"https://10.133.0.52:8000/ready\": dial tcp 10.133.0.52:8000: connect: connection refused" Nov 04 07:41:07.582005 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.581905 5385 generic.go:356] "Generic (PLEG): container finished" podID="128cb6a9-354b-4beb-9ea7-eac997fc81b9" containerID="74dc4f1cc0dcabb75ebf98701f33acbbda5ef4ae35688d7b4ab589fad0f91962" exitCode=0 Nov 04 07:41:07.582462 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.582103 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" event={"ID":"128cb6a9-354b-4beb-9ea7-eac997fc81b9","Type":"ContainerDied","Data":"74dc4f1cc0dcabb75ebf98701f33acbbda5ef4ae35688d7b4ab589fad0f91962"} Nov 04 07:41:07.625552 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.625523 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:41:07.750338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.750308 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-dshm\") pod \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " Nov 04 07:41:07.750577 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.750379 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/128cb6a9-354b-4beb-9ea7-eac997fc81b9-tls-certs\" (UniqueName: \"kubernetes.io/secret/128cb6a9-354b-4beb-9ea7-eac997fc81b9-tls-certs\") pod \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " Nov 04 07:41:07.750577 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.750425 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-home\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-home\") pod \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " Nov 04 07:41:07.750740 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.750577 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-model-cache\") pod \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " Nov 04 07:41:07.750740 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.750615 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/128cb6a9-354b-4beb-9ea7-eac997fc81b9-kube-api-access-btfjq\" (UniqueName: \"kubernetes.io/projected/128cb6a9-354b-4beb-9ea7-eac997fc81b9-kube-api-access-btfjq\") pod \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " Nov 04 07:41:07.750740 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.750645 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-tmp-dir\") pod \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\" (UID: \"128cb6a9-354b-4beb-9ea7-eac997fc81b9\") " Nov 04 07:41:07.750902 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.750742 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-home" pod "128cb6a9-354b-4beb-9ea7-eac997fc81b9" (UID: "128cb6a9-354b-4beb-9ea7-eac997fc81b9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:41:07.750902 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.750829 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-model-cache" pod "128cb6a9-354b-4beb-9ea7-eac997fc81b9" (UID: "128cb6a9-354b-4beb-9ea7-eac997fc81b9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:41:07.751083 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.751049 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-tmp-dir" pod "128cb6a9-354b-4beb-9ea7-eac997fc81b9" (UID: "128cb6a9-354b-4beb-9ea7-eac997fc81b9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:41:07.751327 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.751307 5385 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-model-cache\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:07.751393 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.751335 5385 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-tmp-dir\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:07.751393 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.751353 5385 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-home\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:07.752710 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.752691 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-dshm" pod "128cb6a9-354b-4beb-9ea7-eac997fc81b9" (UID: "128cb6a9-354b-4beb-9ea7-eac997fc81b9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:41:07.752862 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.752840 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/128cb6a9-354b-4beb-9ea7-eac997fc81b9-kube-api-access-btfjq" pod "128cb6a9-354b-4beb-9ea7-eac997fc81b9" (UID: "128cb6a9-354b-4beb-9ea7-eac997fc81b9"). InnerVolumeSpecName "kube-api-access-btfjq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:41:07.752965 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.752948 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/128cb6a9-354b-4beb-9ea7-eac997fc81b9-tls-certs" pod "128cb6a9-354b-4beb-9ea7-eac997fc81b9" (UID: "128cb6a9-354b-4beb-9ea7-eac997fc81b9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:41:07.852243 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.852165 5385 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/128cb6a9-354b-4beb-9ea7-eac997fc81b9-dshm\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:07.852243 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.852194 5385 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/128cb6a9-354b-4beb-9ea7-eac997fc81b9-tls-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:07.852243 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:07.852206 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btfjq\" (UniqueName: \"kubernetes.io/projected/128cb6a9-354b-4beb-9ea7-eac997fc81b9-kube-api-access-btfjq\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:08.202206 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:08.202171 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:08.590242 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:08.590164 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" Nov 04 07:41:08.590242 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:08.590187 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks" event={"ID":"128cb6a9-354b-4beb-9ea7-eac997fc81b9","Type":"ContainerDied","Data":"dd7aa4d36ccb6bc4552e6438e89d8d7f99153cc75d57555398db5707fe684bbe"} Nov 04 07:41:08.590242 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:08.590233 5385 scope.go:122] "RemoveContainer" containerID="74dc4f1cc0dcabb75ebf98701f33acbbda5ef4ae35688d7b4ab589fad0f91962" Nov 04 07:41:08.609854 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:08.609829 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:08.615237 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:08.615215 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks"] Nov 04 07:41:08.617516 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:08.617485 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-29d3ee/premium-w1-29d3ee-kserve-6b8d8df6b7-zdkks"] Nov 04 07:41:08.712847 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:08.712813 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:08.814137 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:08.814101 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="128cb6a9-354b-4beb-9ea7-eac997fc81b9" path="/var/lib/kubelet/pods/128cb6a9-354b-4beb-9ea7-eac997fc81b9/volumes" Nov 04 07:41:08.866961 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:08.866884 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8"] Nov 04 07:41:08.884246 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:08.884205 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf"] Nov 04 07:41:08.964583 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:08.964549 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:09.068469 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:09.068437 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:09.488836 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:09.488799 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:09.594531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:09.594485 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:10.356832 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:10.356788 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:10.476058 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:10.476021 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:11.579069 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:11.579032 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:12.634746 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:12.634713 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:13.587027 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:13.586948 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:13.691791 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:13.691757 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:13.868471 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:13.868288 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" podUID="80160402-c66f-43db-8fd4-91fc30fa1460" containerName="payload-processing" containerID="cri-o://0f000461f6a6771380d8d4927d2af301215d3f224c80f67356dce413896b5222" gracePeriod=25 Nov 04 07:41:13.885798 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:13.885579 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" podUID="a306921b-4523-4b47-915c-2b6251c276fc" containerName="payload-pre-processing" containerID="cri-o://a8796dbe3c2e536c17ebf4231971a8599b1e9c1eaea7ada593dc5042c087f92c" gracePeriod=25 Nov 04 07:41:14.187227 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.187199 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" Nov 04 07:41:14.221964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.221933 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" Nov 04 07:41:14.302483 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.302445 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/80160402-c66f-43db-8fd4-91fc30fa1460-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80160402-c66f-43db-8fd4-91fc30fa1460-plugins-config-volume\") pod \"80160402-c66f-43db-8fd4-91fc30fa1460\" (UID: \"80160402-c66f-43db-8fd4-91fc30fa1460\") " Nov 04 07:41:14.302701 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.302587 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a306921b-4523-4b47-915c-2b6251c276fc-kube-api-access-j5qjp\" (UniqueName: \"kubernetes.io/projected/a306921b-4523-4b47-915c-2b6251c276fc-kube-api-access-j5qjp\") pod \"a306921b-4523-4b47-915c-2b6251c276fc\" (UID: \"a306921b-4523-4b47-915c-2b6251c276fc\") " Nov 04 07:41:14.302701 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.302662 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/80160402-c66f-43db-8fd4-91fc30fa1460-kube-api-access-qstlq\" (UniqueName: \"kubernetes.io/projected/80160402-c66f-43db-8fd4-91fc30fa1460-kube-api-access-qstlq\") pod \"80160402-c66f-43db-8fd4-91fc30fa1460\" (UID: \"80160402-c66f-43db-8fd4-91fc30fa1460\") " Nov 04 07:41:14.302825 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.302811 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a306921b-4523-4b47-915c-2b6251c276fc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a306921b-4523-4b47-915c-2b6251c276fc-plugins-config-volume\") pod \"a306921b-4523-4b47-915c-2b6251c276fc\" (UID: \"a306921b-4523-4b47-915c-2b6251c276fc\") " Nov 04 07:41:14.302880 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.302815 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80160402-c66f-43db-8fd4-91fc30fa1460-plugins-config-volume" pod "80160402-c66f-43db-8fd4-91fc30fa1460" (UID: "80160402-c66f-43db-8fd4-91fc30fa1460"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:41:14.303168 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.303139 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a306921b-4523-4b47-915c-2b6251c276fc-plugins-config-volume" pod "a306921b-4523-4b47-915c-2b6251c276fc" (UID: "a306921b-4523-4b47-915c-2b6251c276fc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:41:14.303311 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.303247 5385 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a306921b-4523-4b47-915c-2b6251c276fc-plugins-config-volume\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:14.303311 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.303259 5385 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80160402-c66f-43db-8fd4-91fc30fa1460-plugins-config-volume\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:14.304824 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.304801 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80160402-c66f-43db-8fd4-91fc30fa1460-kube-api-access-qstlq" pod "80160402-c66f-43db-8fd4-91fc30fa1460" (UID: "80160402-c66f-43db-8fd4-91fc30fa1460"). InnerVolumeSpecName "kube-api-access-qstlq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:41:14.304915 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.304836 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a306921b-4523-4b47-915c-2b6251c276fc-kube-api-access-j5qjp" pod "a306921b-4523-4b47-915c-2b6251c276fc" (UID: "a306921b-4523-4b47-915c-2b6251c276fc"). InnerVolumeSpecName "kube-api-access-j5qjp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:41:14.404743 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.404657 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5qjp\" (UniqueName: \"kubernetes.io/projected/a306921b-4523-4b47-915c-2b6251c276fc-kube-api-access-j5qjp\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:14.404743 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.404690 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qstlq\" (UniqueName: \"kubernetes.io/projected/80160402-c66f-43db-8fd4-91fc30fa1460-kube-api-access-qstlq\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:14.532014 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.531981 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:14.710747 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.710655 5385 generic.go:356] "Generic (PLEG): container finished" podID="a306921b-4523-4b47-915c-2b6251c276fc" containerID="a8796dbe3c2e536c17ebf4231971a8599b1e9c1eaea7ada593dc5042c087f92c" exitCode=0 Nov 04 07:41:14.710747 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.710707 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" Nov 04 07:41:14.710747 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.710738 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" event={"ID":"a306921b-4523-4b47-915c-2b6251c276fc","Type":"ContainerDied","Data":"a8796dbe3c2e536c17ebf4231971a8599b1e9c1eaea7ada593dc5042c087f92c"} Nov 04 07:41:14.711340 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.710794 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf" event={"ID":"a306921b-4523-4b47-915c-2b6251c276fc","Type":"ContainerDied","Data":"6f7e5af367e5c15c0c55454787e5de4a92bb27ca1887a788a755745c8a588bf2"} Nov 04 07:41:14.711340 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.710824 5385 scope.go:122] "RemoveContainer" containerID="a8796dbe3c2e536c17ebf4231971a8599b1e9c1eaea7ada593dc5042c087f92c" Nov 04 07:41:14.712326 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.712299 5385 generic.go:356] "Generic (PLEG): container finished" podID="80160402-c66f-43db-8fd4-91fc30fa1460" containerID="0f000461f6a6771380d8d4927d2af301215d3f224c80f67356dce413896b5222" exitCode=0 Nov 04 07:41:14.712485 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.712356 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" Nov 04 07:41:14.712485 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.712365 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" event={"ID":"80160402-c66f-43db-8fd4-91fc30fa1460","Type":"ContainerDied","Data":"0f000461f6a6771380d8d4927d2af301215d3f224c80f67356dce413896b5222"} Nov 04 07:41:14.712485 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.712409 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8" event={"ID":"80160402-c66f-43db-8fd4-91fc30fa1460","Type":"ContainerDied","Data":"72e68c0b871d1e6ea93770386856e0d1d72a4ab098777f596d65cc2e451d6615"} Nov 04 07:41:14.723028 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.723012 5385 scope.go:122] "RemoveContainer" containerID="a8796dbe3c2e536c17ebf4231971a8599b1e9c1eaea7ada593dc5042c087f92c" Nov 04 07:41:14.723465 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:41:14.723331 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8796dbe3c2e536c17ebf4231971a8599b1e9c1eaea7ada593dc5042c087f92c\": container with ID starting with a8796dbe3c2e536c17ebf4231971a8599b1e9c1eaea7ada593dc5042c087f92c not found: ID does not exist" containerID="a8796dbe3c2e536c17ebf4231971a8599b1e9c1eaea7ada593dc5042c087f92c" Nov 04 07:41:14.723465 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.723363 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8796dbe3c2e536c17ebf4231971a8599b1e9c1eaea7ada593dc5042c087f92c"} err="failed to get container status \"a8796dbe3c2e536c17ebf4231971a8599b1e9c1eaea7ada593dc5042c087f92c\": rpc error: code = NotFound desc = could not find container \"a8796dbe3c2e536c17ebf4231971a8599b1e9c1eaea7ada593dc5042c087f92c\": container with ID starting with a8796dbe3c2e536c17ebf4231971a8599b1e9c1eaea7ada593dc5042c087f92c not found: ID does not exist" Nov 04 07:41:14.723465 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.723387 5385 scope.go:122] "RemoveContainer" containerID="0f000461f6a6771380d8d4927d2af301215d3f224c80f67356dce413896b5222" Nov 04 07:41:14.733001 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.732920 5385 scope.go:122] "RemoveContainer" containerID="0f000461f6a6771380d8d4927d2af301215d3f224c80f67356dce413896b5222" Nov 04 07:41:14.733221 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:41:14.733202 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f000461f6a6771380d8d4927d2af301215d3f224c80f67356dce413896b5222\": container with ID starting with 0f000461f6a6771380d8d4927d2af301215d3f224c80f67356dce413896b5222 not found: ID does not exist" containerID="0f000461f6a6771380d8d4927d2af301215d3f224c80f67356dce413896b5222" Nov 04 07:41:14.733264 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.733229 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f000461f6a6771380d8d4927d2af301215d3f224c80f67356dce413896b5222"} err="failed to get container status \"0f000461f6a6771380d8d4927d2af301215d3f224c80f67356dce413896b5222\": rpc error: code = NotFound desc = could not find container \"0f000461f6a6771380d8d4927d2af301215d3f224c80f67356dce413896b5222\": container with ID starting with 0f000461f6a6771380d8d4927d2af301215d3f224c80f67356dce413896b5222 not found: ID does not exist" Nov 04 07:41:14.735298 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.735276 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf"] Nov 04 07:41:14.738991 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.738971 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-3da894-6f55f8f687-8v2pf"] Nov 04 07:41:14.750589 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.750566 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8"] Nov 04 07:41:14.756711 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.756689 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-3da894-966b64c9d-vc6p8"] Nov 04 07:41:14.823139 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.823098 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="80160402-c66f-43db-8fd4-91fc30fa1460" path="/var/lib/kubelet/pods/80160402-c66f-43db-8fd4-91fc30fa1460/volumes" Nov 04 07:41:14.823497 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:14.823483 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a306921b-4523-4b47-915c-2b6251c276fc" path="/var/lib/kubelet/pods/a306921b-4523-4b47-915c-2b6251c276fc/volumes" Nov 04 07:41:15.392153 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:15.392114 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:16.276252 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:16.276216 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:16.551145 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:16.551050 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:16.969232 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:16.969194 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:17.169648 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:17.169611 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:17.606307 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:17.606277 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:18.237622 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.237586 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb"] Nov 04 07:41:18.237889 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.237837 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" podUID="3af2b4e6-98e9-422b-97b1-e2e2f1ea5909" containerName="maas-api" containerID="cri-o://21e999625e0ae2335a80f4aa981cd264673b8ecdb434e60b54a1acf2019cb5ee" gracePeriod=30 Nov 04 07:41:18.273696 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.273669 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:18.568580 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.567657 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:18.698801 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.698778 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:41:18.743104 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.743073 5385 generic.go:356] "Generic (PLEG): container finished" podID="3af2b4e6-98e9-422b-97b1-e2e2f1ea5909" containerID="21e999625e0ae2335a80f4aa981cd264673b8ecdb434e60b54a1acf2019cb5ee" exitCode=0 Nov 04 07:41:18.743304 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.743128 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" Nov 04 07:41:18.743304 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.743169 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" event={"ID":"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909","Type":"ContainerDied","Data":"21e999625e0ae2335a80f4aa981cd264673b8ecdb434e60b54a1acf2019cb5ee"} Nov 04 07:41:18.743304 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.743220 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" event={"ID":"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909","Type":"ContainerDied","Data":"1964b60a55904ebf47ceb40ea0c73d493495455168f48a806aabaa048cb17d83"} Nov 04 07:41:18.743304 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.743246 5385 scope.go:122] "RemoveContainer" containerID="21e999625e0ae2335a80f4aa981cd264673b8ecdb434e60b54a1acf2019cb5ee" Nov 04 07:41:18.754890 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.754868 5385 scope.go:122] "RemoveContainer" containerID="21e999625e0ae2335a80f4aa981cd264673b8ecdb434e60b54a1acf2019cb5ee" Nov 04 07:41:18.755181 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:41:18.755161 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21e999625e0ae2335a80f4aa981cd264673b8ecdb434e60b54a1acf2019cb5ee\": container with ID starting with 21e999625e0ae2335a80f4aa981cd264673b8ecdb434e60b54a1acf2019cb5ee not found: ID does not exist" containerID="21e999625e0ae2335a80f4aa981cd264673b8ecdb434e60b54a1acf2019cb5ee" Nov 04 07:41:18.755261 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.755191 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21e999625e0ae2335a80f4aa981cd264673b8ecdb434e60b54a1acf2019cb5ee"} err="failed to get container status \"21e999625e0ae2335a80f4aa981cd264673b8ecdb434e60b54a1acf2019cb5ee\": rpc error: code = NotFound desc = could not find container \"21e999625e0ae2335a80f4aa981cd264673b8ecdb434e60b54a1acf2019cb5ee\": container with ID starting with 21e999625e0ae2335a80f4aa981cd264673b8ecdb434e60b54a1acf2019cb5ee not found: ID does not exist" Nov 04 07:41:18.816820 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.816789 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-kube-api-access-dbc89\" (UniqueName: \"kubernetes.io/projected/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-kube-api-access-dbc89\") pod \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\" (UID: \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\") " Nov 04 07:41:18.816955 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.816827 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-metrics-certs\") pod \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\" (UID: \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\") " Nov 04 07:41:18.816955 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.816888 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-maas-api-tls\") pod \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\" (UID: \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\") " Nov 04 07:41:18.816955 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.816920 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-openshift-service-ca\") pod \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\" (UID: \"3af2b4e6-98e9-422b-97b1-e2e2f1ea5909\") " Nov 04 07:41:18.817365 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.817333 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-openshift-service-ca" pod "3af2b4e6-98e9-422b-97b1-e2e2f1ea5909" (UID: "3af2b4e6-98e9-422b-97b1-e2e2f1ea5909"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:41:18.817530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.817493 5385 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-openshift-service-ca\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:18.819023 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.818973 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-metrics-certs" pod "3af2b4e6-98e9-422b-97b1-e2e2f1ea5909" (UID: "3af2b4e6-98e9-422b-97b1-e2e2f1ea5909"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:41:18.819102 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.819027 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-maas-api-tls" pod "3af2b4e6-98e9-422b-97b1-e2e2f1ea5909" (UID: "3af2b4e6-98e9-422b-97b1-e2e2f1ea5909"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:41:18.819102 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.819047 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-kube-api-access-dbc89" pod "3af2b4e6-98e9-422b-97b1-e2e2f1ea5909" (UID: "3af2b4e6-98e9-422b-97b1-e2e2f1ea5909"). InnerVolumeSpecName "kube-api-access-dbc89". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:41:18.918794 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.918759 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbc89\" (UniqueName: \"kubernetes.io/projected/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-kube-api-access-dbc89\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:18.918794 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.918791 5385 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-metrics-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:18.918794 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:18.918803 5385 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909-maas-api-tls\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:19.092955 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:19.092880 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb"] Nov 04 07:41:19.100865 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:19.100832 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb"] Nov 04 07:41:19.519372 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:19.519334 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:20.535907 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:20.535639 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:20.680527 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:20.680053 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:20.792607 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:20.792405 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:20.824635 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:20.824601 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3af2b4e6-98e9-422b-97b1-e2e2f1ea5909" path="/var/lib/kubelet/pods/3af2b4e6-98e9-422b-97b1-e2e2f1ea5909/volumes" Nov 04 07:41:20.901191 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:20.901136 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.108789 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-9699b4dc9-g954t"] Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109730 5385 cpu_manager.go:414] "containerMap: removing container" podUID="808ce2ba-76f7-4825-986d-4d1030e4737e" containerName="payload-processing" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109752 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="808ce2ba-76f7-4825-986d-4d1030e4737e" containerName="payload-processing" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109769 5385 cpu_manager.go:414] "containerMap: removing container" podUID="cb02a450-1430-46b8-95bd-94edf3e27328" containerName="istio-proxy" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109776 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb02a450-1430-46b8-95bd-94edf3e27328" containerName="istio-proxy" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109792 5385 cpu_manager.go:414] "containerMap: removing container" podUID="6b7e3053-63a4-43c2-ada7-f98132ef6d4d" containerName="istio-proxy" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109801 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b7e3053-63a4-43c2-ada7-f98132ef6d4d" containerName="istio-proxy" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109826 5385 cpu_manager.go:414] "containerMap: removing container" podUID="6b7e3053-63a4-43c2-ada7-f98132ef6d4d" containerName="istio-proxy" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109833 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b7e3053-63a4-43c2-ada7-f98132ef6d4d" containerName="istio-proxy" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109854 5385 cpu_manager.go:414] "containerMap: removing container" podUID="128cb6a9-354b-4beb-9ea7-eac997fc81b9" containerName="main" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109861 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="128cb6a9-354b-4beb-9ea7-eac997fc81b9" containerName="main" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109872 5385 cpu_manager.go:414] "containerMap: removing container" podUID="e977f4f4-c3aa-484d-b64a-fda4d19ce7de" containerName="istio-proxy" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109879 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e977f4f4-c3aa-484d-b64a-fda4d19ce7de" containerName="istio-proxy" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109903 5385 cpu_manager.go:414] "containerMap: removing container" podUID="b3a8327a-27f1-4947-8567-49372cc916ec" containerName="payload-pre-processing" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109912 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b3a8327a-27f1-4947-8567-49372cc916ec" containerName="payload-pre-processing" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109923 5385 cpu_manager.go:414] "containerMap: removing container" podUID="947a11da-cef0-4a9c-8e51-568588b5885a" containerName="payload-processing" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109930 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="947a11da-cef0-4a9c-8e51-568588b5885a" containerName="payload-processing" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109955 5385 cpu_manager.go:414] "containerMap: removing container" podUID="f6e3296f-cbf9-43d0-8a3f-86e6d0190b63" containerName="maas-api" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109964 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6e3296f-cbf9-43d0-8a3f-86e6d0190b63" containerName="maas-api" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109974 5385 cpu_manager.go:414] "containerMap: removing container" podUID="3af2b4e6-98e9-422b-97b1-e2e2f1ea5909" containerName="maas-api" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109981 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3af2b4e6-98e9-422b-97b1-e2e2f1ea5909" containerName="maas-api" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.109998 5385 cpu_manager.go:414] "containerMap: removing container" podUID="80160402-c66f-43db-8fd4-91fc30fa1460" containerName="payload-processing" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.110005 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="80160402-c66f-43db-8fd4-91fc30fa1460" containerName="payload-processing" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.110019 5385 cpu_manager.go:414] "containerMap: removing container" podUID="a306921b-4523-4b47-915c-2b6251c276fc" containerName="payload-pre-processing" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.110026 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a306921b-4523-4b47-915c-2b6251c276fc" containerName="payload-pre-processing" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.110144 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb02a450-1430-46b8-95bd-94edf3e27328" containerName="istio-proxy" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.110157 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b7e3053-63a4-43c2-ada7-f98132ef6d4d" containerName="istio-proxy" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.110170 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b7e3053-63a4-43c2-ada7-f98132ef6d4d" containerName="istio-proxy" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.110181 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="128cb6a9-354b-4beb-9ea7-eac997fc81b9" containerName="main" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.110191 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="e977f4f4-c3aa-484d-b64a-fda4d19ce7de" containerName="istio-proxy" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.110201 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="b3a8327a-27f1-4947-8567-49372cc916ec" containerName="payload-pre-processing" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.110211 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="947a11da-cef0-4a9c-8e51-568588b5885a" containerName="payload-processing" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.110223 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6e3296f-cbf9-43d0-8a3f-86e6d0190b63" containerName="maas-api" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.110233 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="3af2b4e6-98e9-422b-97b1-e2e2f1ea5909" containerName="maas-api" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.110243 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="80160402-c66f-43db-8fd4-91fc30fa1460" containerName="payload-processing" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.110251 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="a306921b-4523-4b47-915c-2b6251c276fc" containerName="payload-pre-processing" Nov 04 07:41:21.112126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.110261 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="808ce2ba-76f7-4825-986d-4d1030e4737e" containerName="payload-processing" Nov 04 07:41:21.115678 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.115652 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:21.122037 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.121956 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-c74sz\"" Nov 04 07:41:21.124437 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.124375 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-9699b4dc9-g954t"] Nov 04 07:41:21.125196 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.124974 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\"" Nov 04 07:41:21.125196 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.125004 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-5cc1fb4bfccd\"" Nov 04 07:41:21.125196 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.125083 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\"" Nov 04 07:41:21.125196 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.125146 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\"" Nov 04 07:41:21.125533 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.125208 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\"" Nov 04 07:41:21.220251 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.220216 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a112a5b6-387a-48c0-9d0a-d437222311ea-provider-credentials\") pod \"payload-processing-external-model-9699b4dc9-g954t\" (UID: \"a112a5b6-387a-48c0-9d0a-d437222311ea\") " pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:21.220461 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.220375 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a112a5b6-387a-48c0-9d0a-d437222311ea-routing-overlay-volume\") pod \"payload-processing-external-model-9699b4dc9-g954t\" (UID: \"a112a5b6-387a-48c0-9d0a-d437222311ea\") " pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:21.220461 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.220394 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sttpb\" (UniqueName: \"kubernetes.io/projected/a112a5b6-387a-48c0-9d0a-d437222311ea-kube-api-access-sttpb\") pod \"payload-processing-external-model-9699b4dc9-g954t\" (UID: \"a112a5b6-387a-48c0-9d0a-d437222311ea\") " pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:21.220622 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.220557 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a112a5b6-387a-48c0-9d0a-d437222311ea-plugins-config-volume\") pod \"payload-processing-external-model-9699b4dc9-g954t\" (UID: \"a112a5b6-387a-48c0-9d0a-d437222311ea\") " pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:21.321584 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.321542 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a112a5b6-387a-48c0-9d0a-d437222311ea-plugins-config-volume\") pod \"payload-processing-external-model-9699b4dc9-g954t\" (UID: \"a112a5b6-387a-48c0-9d0a-d437222311ea\") " pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:21.321788 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.321637 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a112a5b6-387a-48c0-9d0a-d437222311ea-provider-credentials\") pod \"payload-processing-external-model-9699b4dc9-g954t\" (UID: \"a112a5b6-387a-48c0-9d0a-d437222311ea\") " pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:21.321788 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.321733 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a112a5b6-387a-48c0-9d0a-d437222311ea-routing-overlay-volume\") pod \"payload-processing-external-model-9699b4dc9-g954t\" (UID: \"a112a5b6-387a-48c0-9d0a-d437222311ea\") " pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:21.321788 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.321757 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sttpb\" (UniqueName: \"kubernetes.io/projected/a112a5b6-387a-48c0-9d0a-d437222311ea-kube-api-access-sttpb\") pod \"payload-processing-external-model-9699b4dc9-g954t\" (UID: \"a112a5b6-387a-48c0-9d0a-d437222311ea\") " pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:21.322531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.322480 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a112a5b6-387a-48c0-9d0a-d437222311ea-routing-overlay-volume\") pod \"payload-processing-external-model-9699b4dc9-g954t\" (UID: \"a112a5b6-387a-48c0-9d0a-d437222311ea\") " pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:21.322659 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.322608 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a112a5b6-387a-48c0-9d0a-d437222311ea-plugins-config-volume\") pod \"payload-processing-external-model-9699b4dc9-g954t\" (UID: \"a112a5b6-387a-48c0-9d0a-d437222311ea\") " pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:21.324449 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.324420 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a112a5b6-387a-48c0-9d0a-d437222311ea-provider-credentials\") pod \"payload-processing-external-model-9699b4dc9-g954t\" (UID: \"a112a5b6-387a-48c0-9d0a-d437222311ea\") " pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:21.335277 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.335243 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sttpb\" (UniqueName: \"kubernetes.io/projected/a112a5b6-387a-48c0-9d0a-d437222311ea-kube-api-access-sttpb\") pod \"payload-processing-external-model-9699b4dc9-g954t\" (UID: \"a112a5b6-387a-48c0-9d0a-d437222311ea\") " pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:21.453996 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.453883 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:21.643439 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.642071 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:21.689792 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.689737 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-3da894-5c8c8cc75-gdnvb" podUID="3af2b4e6-98e9-422b-97b1-e2e2f1ea5909" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.67:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 04 07:41:21.827045 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:21.822248 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-9699b4dc9-g954t"] Nov 04 07:41:21.831880 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:41:21.831833 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda112a5b6_387a_48c0_9d0a_d437222311ea.slice/crio-b7ede3ea6017a6be234ee44b797f65755f078f9793be3aaea89c7bbe82adb44c WatchSource:0}: Error finding container b7ede3ea6017a6be234ee44b797f65755f078f9793be3aaea89c7bbe82adb44c: Status 404 returned error can't find the container with id b7ede3ea6017a6be234ee44b797f65755f078f9793be3aaea89c7bbe82adb44c Nov 04 07:41:22.517165 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:22.516950 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:22.626732 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:22.626699 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:22.836451 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:22.836363 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" event={"ID":"a112a5b6-387a-48c0-9d0a-d437222311ea","Type":"ContainerStarted","Data":"962c525bce62da85fb259b7a8c7898dd1d46feb834aca7cfb7cf6816836dcc0e"} Nov 04 07:41:22.836451 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:22.836420 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" event={"ID":"a112a5b6-387a-48c0-9d0a-d437222311ea","Type":"ContainerStarted","Data":"b7ede3ea6017a6be234ee44b797f65755f078f9793be3aaea89c7bbe82adb44c"} Nov 04 07:41:22.836924 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:22.836461 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:23.534472 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:23.534433 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:23.803401 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:23.803315 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:23.943187 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:23.943129 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:24.141787 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:24.141751 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:24.538983 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:24.538878 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:25.199405 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:25.199371 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:25.325699 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:25.325667 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:25.978993 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:25.978953 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:26.953926 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:26.953880 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:27.585882 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:27.585821 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" podStartSLOduration=6.585805569 podStartE2EDuration="6.585805569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:41:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:41:22.879779012 +0000 UTC m=+1518.213718597" watchObservedRunningTime="2026-11-04 07:41:27.585805569 +0000 UTC m=+1522.919745153" Nov 04 07:41:27.586225 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:27.586204 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt"] Nov 04 07:41:27.676059 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:27.676028 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:27.777783 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:27.777747 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:27.884830 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:27.884770 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:28.683470 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:28.683429 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:29.214261 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:29.214228 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:29.364274 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:29.364201 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:29.503887 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:29.503793 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:29.809445 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:29.809354 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:30.679498 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:30.679458 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:30.895203 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:41:30.895168 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:41:31.790402 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:31.790366 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:31.894786 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:31.894617 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:32.179957 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:32.179917 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:32.573619 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:32.573299 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p"] Nov 04 07:41:32.583283 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:32.578724 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" podUID="5db0f011-9979-4ecd-9525-5a02cfd6c8c8" containerName="istio-proxy" containerID="cri-o://0c30904cec1443e4ac89ecc3024848d6230989a4308a3b02c8cb69986f8a80a6" gracePeriod=30 Nov 04 07:41:32.588562 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:32.587763 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" podUID="5c14cd7f-ad7e-446c-be34-1b251628c970" containerName="payload-processing" containerID="cri-o://f759f2df6e1d3800dbf6af939df5945a3b27ae6fa3d1b64a26790b0be33dfd22" gracePeriod=25 Nov 04 07:41:32.592098 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:32.591006 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:32.727335 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:32.727300 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:32.837020 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:32.836928 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:33.031525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:33.029213 5385 patch_prober.go:29] interesting pod/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 07:41:33.031525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:33.029291 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" podUID="5db0f011-9979-4ecd-9525-5a02cfd6c8c8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 07:41:33.043606 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:33.043566 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:33.302311 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:33.302251 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:33.857721 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:33.857686 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:34.503254 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:34.503218 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:35.619988 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:35.619952 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:35.808567 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:35.808520 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:36.413023 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:36.412205 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t"] Nov 04 07:41:36.413535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:36.413468 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" podUID="34fb6ab3-7f51-47c3-8d8d-dd057d054eb0" containerName="maas-api" containerID="cri-o://f8e0458a074079f228a7c4bf649b1da03b827faa22fe6eace4896997c942c02f" gracePeriod=30 Nov 04 07:41:36.460092 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:36.460056 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:36.616201 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:36.616168 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:36.708221 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:36.708196 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:41:36.796736 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:36.796705 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-maas-api-tls\") pod \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\" (UID: \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\") " Nov 04 07:41:36.796924 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:36.796784 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-kube-api-access-z45bf\" (UniqueName: \"kubernetes.io/projected/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-kube-api-access-z45bf\") pod \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\" (UID: \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\") " Nov 04 07:41:36.796924 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:36.796820 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-openshift-service-ca\") pod \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\" (UID: \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\") " Nov 04 07:41:36.796924 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:36.796842 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-metrics-certs\") pod \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\" (UID: \"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0\") " Nov 04 07:41:36.797206 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:36.797178 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-openshift-service-ca" pod "34fb6ab3-7f51-47c3-8d8d-dd057d054eb0" (UID: "34fb6ab3-7f51-47c3-8d8d-dd057d054eb0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:41:36.798946 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:36.798917 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-kube-api-access-z45bf" pod "34fb6ab3-7f51-47c3-8d8d-dd057d054eb0" (UID: "34fb6ab3-7f51-47c3-8d8d-dd057d054eb0"). InnerVolumeSpecName "kube-api-access-z45bf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:41:36.799086 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:36.799005 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-maas-api-tls" pod "34fb6ab3-7f51-47c3-8d8d-dd057d054eb0" (UID: "34fb6ab3-7f51-47c3-8d8d-dd057d054eb0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:41:36.799086 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:36.799035 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-metrics-certs" pod "34fb6ab3-7f51-47c3-8d8d-dd057d054eb0" (UID: "34fb6ab3-7f51-47c3-8d8d-dd057d054eb0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:41:36.903581 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:36.899087 5385 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-maas-api-tls\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:36.903581 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:36.899135 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z45bf\" (UniqueName: \"kubernetes.io/projected/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-kube-api-access-z45bf\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:36.903581 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:36.899154 5385 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-openshift-service-ca\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:36.903581 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:36.899168 5385 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0-metrics-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:37.084873 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:37.084777 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:37.181573 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:37.181475 5385 generic.go:356] "Generic (PLEG): container finished" podID="34fb6ab3-7f51-47c3-8d8d-dd057d054eb0" containerID="f8e0458a074079f228a7c4bf649b1da03b827faa22fe6eace4896997c942c02f" exitCode=0 Nov 04 07:41:37.181573 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:37.181557 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" event={"ID":"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0","Type":"ContainerDied","Data":"f8e0458a074079f228a7c4bf649b1da03b827faa22fe6eace4896997c942c02f"} Nov 04 07:41:37.181833 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:37.181630 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" event={"ID":"34fb6ab3-7f51-47c3-8d8d-dd057d054eb0","Type":"ContainerDied","Data":"022beb131787804b0fd1665d6d1cd74b7ca0cbb9e556307250ea5d9152d2ac64"} Nov 04 07:41:37.181833 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:37.181567 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t" Nov 04 07:41:37.181833 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:37.181661 5385 scope.go:122] "RemoveContainer" containerID="f8e0458a074079f228a7c4bf649b1da03b827faa22fe6eace4896997c942c02f" Nov 04 07:41:37.191398 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:37.191377 5385 scope.go:122] "RemoveContainer" containerID="f8e0458a074079f228a7c4bf649b1da03b827faa22fe6eace4896997c942c02f" Nov 04 07:41:37.191722 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:41:37.191700 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8e0458a074079f228a7c4bf649b1da03b827faa22fe6eace4896997c942c02f\": container with ID starting with f8e0458a074079f228a7c4bf649b1da03b827faa22fe6eace4896997c942c02f not found: ID does not exist" containerID="f8e0458a074079f228a7c4bf649b1da03b827faa22fe6eace4896997c942c02f" Nov 04 07:41:37.191830 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:37.191735 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8e0458a074079f228a7c4bf649b1da03b827faa22fe6eace4896997c942c02f"} err="failed to get container status \"f8e0458a074079f228a7c4bf649b1da03b827faa22fe6eace4896997c942c02f\": rpc error: code = NotFound desc = could not find container \"f8e0458a074079f228a7c4bf649b1da03b827faa22fe6eace4896997c942c02f\": container with ID starting with f8e0458a074079f228a7c4bf649b1da03b827faa22fe6eace4896997c942c02f not found: ID does not exist" Nov 04 07:41:37.200655 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:37.200631 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t"] Nov 04 07:41:37.205356 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:37.205333 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-be2a74-ddc75664-c9d4t"] Nov 04 07:41:37.948965 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:37.948931 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:41:37.985912 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:37.985849 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:38.027528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.025542 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-podinfo\") pod \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " Nov 04 07:41:38.027528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.025627 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-workload-certs\") pod \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " Nov 04 07:41:38.027528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.025660 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-envoy\") pod \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " Nov 04 07:41:38.027528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.025716 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-token\" (UniqueName: \"kubernetes.io/projected/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-token\") pod \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " Nov 04 07:41:38.027528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.025742 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-kube-api-access-mxklf\" (UniqueName: \"kubernetes.io/projected/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-kube-api-access-mxklf\") pod \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " Nov 04 07:41:38.027528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.025778 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-workload-socket\") pod \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " Nov 04 07:41:38.027528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.025840 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istiod-ca-cert\") pod \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " Nov 04 07:41:38.027528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.025953 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-credential-socket\") pod \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " Nov 04 07:41:38.027528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.025993 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-data\") pod \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\" (UID: \"5db0f011-9979-4ecd-9525-5a02cfd6c8c8\") " Nov 04 07:41:38.043560 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.043049 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-data" pod "5db0f011-9979-4ecd-9525-5a02cfd6c8c8" (UID: "5db0f011-9979-4ecd-9525-5a02cfd6c8c8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:41:38.047540 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.046406 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istiod-ca-cert" pod "5db0f011-9979-4ecd-9525-5a02cfd6c8c8" (UID: "5db0f011-9979-4ecd-9525-5a02cfd6c8c8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:41:38.072725 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.057611 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-workload-socket" pod "5db0f011-9979-4ecd-9525-5a02cfd6c8c8" (UID: "5db0f011-9979-4ecd-9525-5a02cfd6c8c8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:41:38.072725 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.058324 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-podinfo" pod "5db0f011-9979-4ecd-9525-5a02cfd6c8c8" (UID: "5db0f011-9979-4ecd-9525-5a02cfd6c8c8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 07:41:38.072725 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.058567 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-credential-socket" pod "5db0f011-9979-4ecd-9525-5a02cfd6c8c8" (UID: "5db0f011-9979-4ecd-9525-5a02cfd6c8c8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:41:38.072725 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.058742 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-workload-certs" pod "5db0f011-9979-4ecd-9525-5a02cfd6c8c8" (UID: "5db0f011-9979-4ecd-9525-5a02cfd6c8c8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:41:38.096086 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.089794 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-kube-api-access-mxklf" pod "5db0f011-9979-4ecd-9525-5a02cfd6c8c8" (UID: "5db0f011-9979-4ecd-9525-5a02cfd6c8c8"). InnerVolumeSpecName "kube-api-access-mxklf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:41:38.096086 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.089881 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-token" pod "5db0f011-9979-4ecd-9525-5a02cfd6c8c8" (UID: "5db0f011-9979-4ecd-9525-5a02cfd6c8c8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:41:38.096086 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.089949 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-envoy" pod "5db0f011-9979-4ecd-9525-5a02cfd6c8c8" (UID: "5db0f011-9979-4ecd-9525-5a02cfd6c8c8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:41:38.127583 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.127492 5385 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-podinfo\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:38.127583 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.127587 5385 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-workload-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:38.127788 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.127598 5385 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-envoy\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:38.127788 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.127607 5385 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-token\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:38.127788 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.127617 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxklf\" (UniqueName: \"kubernetes.io/projected/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-kube-api-access-mxklf\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:38.127788 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.127626 5385 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-workload-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:38.127788 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.127634 5385 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istiod-ca-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:38.127788 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.127643 5385 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-credential-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:38.127788 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.127651 5385 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5db0f011-9979-4ecd-9525-5a02cfd6c8c8-istio-data\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:38.189285 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.189198 5385 generic.go:356] "Generic (PLEG): container finished" podID="5db0f011-9979-4ecd-9525-5a02cfd6c8c8" containerID="0c30904cec1443e4ac89ecc3024848d6230989a4308a3b02c8cb69986f8a80a6" exitCode=0 Nov 04 07:41:38.189285 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.189251 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" Nov 04 07:41:38.189493 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.189286 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" event={"ID":"5db0f011-9979-4ecd-9525-5a02cfd6c8c8","Type":"ContainerDied","Data":"0c30904cec1443e4ac89ecc3024848d6230989a4308a3b02c8cb69986f8a80a6"} Nov 04 07:41:38.189493 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.189334 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p" event={"ID":"5db0f011-9979-4ecd-9525-5a02cfd6c8c8","Type":"ContainerDied","Data":"ba52d0ef48ab4848fa3d37dac245bb42c829f5c2c864394b61d6b499b2c69dc6"} Nov 04 07:41:38.189493 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.189355 5385 scope.go:122] "RemoveContainer" containerID="0c30904cec1443e4ac89ecc3024848d6230989a4308a3b02c8cb69986f8a80a6" Nov 04 07:41:38.199015 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.198994 5385 scope.go:122] "RemoveContainer" containerID="0c30904cec1443e4ac89ecc3024848d6230989a4308a3b02c8cb69986f8a80a6" Nov 04 07:41:38.199292 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:41:38.199275 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c30904cec1443e4ac89ecc3024848d6230989a4308a3b02c8cb69986f8a80a6\": container with ID starting with 0c30904cec1443e4ac89ecc3024848d6230989a4308a3b02c8cb69986f8a80a6 not found: ID does not exist" containerID="0c30904cec1443e4ac89ecc3024848d6230989a4308a3b02c8cb69986f8a80a6" Nov 04 07:41:38.199334 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.199302 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c30904cec1443e4ac89ecc3024848d6230989a4308a3b02c8cb69986f8a80a6"} err="failed to get container status \"0c30904cec1443e4ac89ecc3024848d6230989a4308a3b02c8cb69986f8a80a6\": rpc error: code = NotFound desc = could not find container \"0c30904cec1443e4ac89ecc3024848d6230989a4308a3b02c8cb69986f8a80a6\": container with ID starting with 0c30904cec1443e4ac89ecc3024848d6230989a4308a3b02c8cb69986f8a80a6 not found: ID does not exist" Nov 04 07:41:38.216817 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.216791 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p"] Nov 04 07:41:38.221028 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.221003 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-29d3ee-openshift-default-844cdd95d7-7hh6p"] Nov 04 07:41:38.707549 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.707467 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:38.815023 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.814990 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34fb6ab3-7f51-47c3-8d8d-dd057d054eb0" path="/var/lib/kubelet/pods/34fb6ab3-7f51-47c3-8d8d-dd057d054eb0/volumes" Nov 04 07:41:38.815476 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.815461 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5db0f011-9979-4ecd-9525-5a02cfd6c8c8" path="/var/lib/kubelet/pods/5db0f011-9979-4ecd-9525-5a02cfd6c8c8/volumes" Nov 04 07:41:38.860538 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:38.849491 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:39.789178 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:39.789145 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:39.959091 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:39.959017 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:40.079398 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:40.079316 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:40.441062 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:40.441027 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:40.732014 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:40.731920 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:40.751781 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:40.751741 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b"] Nov 04 07:41:40.752017 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:40.751992 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" podUID="e8a5b7b3-360d-475c-87b7-6a845394a09c" containerName="istio-proxy" containerID="cri-o://0d1de9dde96bc674c1447ee6124002858562b9ebf1c29bd9fc42c55b2cc9f476" gracePeriod=30 Nov 04 07:41:40.831991 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:40.831958 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:41.264487 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:41.264454 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:41.720957 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:41.720908 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:42.244624 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:42.244588 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" podUID="5c14cd7f-ad7e-446c-be34-1b251628c970" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 07:41:42.444826 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:42.444788 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:42.549163 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:42.549063 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:42.981878 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:42.981843 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:43.365555 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:43.365450 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:43.663396 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:43.663367 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:44.158444 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:44.158409 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:44.475206 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:44.475105 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:45.376742 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:45.376710 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:46.022789 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.022726 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:41:46.064321 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.064290 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-data\") pod \"e8a5b7b3-360d-475c-87b7-6a845394a09c\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " Nov 04 07:41:46.064522 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.064405 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-token\" (UniqueName: \"kubernetes.io/projected/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-token\") pod \"e8a5b7b3-360d-475c-87b7-6a845394a09c\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " Nov 04 07:41:46.064522 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.064452 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-podinfo\") pod \"e8a5b7b3-360d-475c-87b7-6a845394a09c\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " Nov 04 07:41:46.064522 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.064500 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e8a5b7b3-360d-475c-87b7-6a845394a09c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8a5b7b3-360d-475c-87b7-6a845394a09c-istiod-ca-cert\") pod \"e8a5b7b3-360d-475c-87b7-6a845394a09c\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " Nov 04 07:41:46.064691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.064547 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-credential-socket\") pod \"e8a5b7b3-360d-475c-87b7-6a845394a09c\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " Nov 04 07:41:46.064691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.064580 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-data" pod "e8a5b7b3-360d-475c-87b7-6a845394a09c" (UID: "e8a5b7b3-360d-475c-87b7-6a845394a09c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:41:46.064691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.064617 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-envoy\") pod \"e8a5b7b3-360d-475c-87b7-6a845394a09c\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " Nov 04 07:41:46.064843 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.064697 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8a5b7b3-360d-475c-87b7-6a845394a09c-kube-api-access-kgkkp\" (UniqueName: \"kubernetes.io/projected/e8a5b7b3-360d-475c-87b7-6a845394a09c-kube-api-access-kgkkp\") pod \"e8a5b7b3-360d-475c-87b7-6a845394a09c\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " Nov 04 07:41:46.064843 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.064759 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-workload-socket\") pod \"e8a5b7b3-360d-475c-87b7-6a845394a09c\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " Nov 04 07:41:46.064843 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.064792 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-workload-certs\") pod \"e8a5b7b3-360d-475c-87b7-6a845394a09c\" (UID: \"e8a5b7b3-360d-475c-87b7-6a845394a09c\") " Nov 04 07:41:46.065582 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.065530 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-workload-socket" pod "e8a5b7b3-360d-475c-87b7-6a845394a09c" (UID: "e8a5b7b3-360d-475c-87b7-6a845394a09c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:41:46.065713 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.065629 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-credential-socket" pod "e8a5b7b3-360d-475c-87b7-6a845394a09c" (UID: "e8a5b7b3-360d-475c-87b7-6a845394a09c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:41:46.065815 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.065783 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-workload-certs" pod "e8a5b7b3-360d-475c-87b7-6a845394a09c" (UID: "e8a5b7b3-360d-475c-87b7-6a845394a09c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:41:46.065926 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.065811 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8a5b7b3-360d-475c-87b7-6a845394a09c-istiod-ca-cert" pod "e8a5b7b3-360d-475c-87b7-6a845394a09c" (UID: "e8a5b7b3-360d-475c-87b7-6a845394a09c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:41:46.065995 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.065977 5385 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-workload-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:46.066048 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.065997 5385 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-workload-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:46.066048 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.066012 5385 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-data\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:46.066048 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.066027 5385 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8a5b7b3-360d-475c-87b7-6a845394a09c-istiod-ca-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:46.066048 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.066043 5385 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-credential-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:46.067705 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.067678 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-podinfo" pod "e8a5b7b3-360d-475c-87b7-6a845394a09c" (UID: "e8a5b7b3-360d-475c-87b7-6a845394a09c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 07:41:46.071839 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.071810 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-envoy" pod "e8a5b7b3-360d-475c-87b7-6a845394a09c" (UID: "e8a5b7b3-360d-475c-87b7-6a845394a09c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:41:46.071950 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.071848 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8a5b7b3-360d-475c-87b7-6a845394a09c-kube-api-access-kgkkp" pod "e8a5b7b3-360d-475c-87b7-6a845394a09c" (UID: "e8a5b7b3-360d-475c-87b7-6a845394a09c"). InnerVolumeSpecName "kube-api-access-kgkkp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:41:46.073045 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.073020 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-token" pod "e8a5b7b3-360d-475c-87b7-6a845394a09c" (UID: "e8a5b7b3-360d-475c-87b7-6a845394a09c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:41:46.167429 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.167395 5385 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-token\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:46.167429 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.167424 5385 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-podinfo\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:46.167429 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.167434 5385 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8a5b7b3-360d-475c-87b7-6a845394a09c-istio-envoy\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:46.167702 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.167447 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgkkp\" (UniqueName: \"kubernetes.io/projected/e8a5b7b3-360d-475c-87b7-6a845394a09c-kube-api-access-kgkkp\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:46.229531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.227041 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g"] Nov 04 07:41:46.235679 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.235637 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n"] Nov 04 07:41:46.241967 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.241938 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l"] Nov 04 07:41:46.246871 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.246837 5385 generic.go:356] "Generic (PLEG): container finished" podID="e8a5b7b3-360d-475c-87b7-6a845394a09c" containerID="0d1de9dde96bc674c1447ee6124002858562b9ebf1c29bd9fc42c55b2cc9f476" exitCode=0 Nov 04 07:41:46.247016 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.246906 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" Nov 04 07:41:46.247016 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.246913 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" event={"ID":"e8a5b7b3-360d-475c-87b7-6a845394a09c","Type":"ContainerDied","Data":"0d1de9dde96bc674c1447ee6124002858562b9ebf1c29bd9fc42c55b2cc9f476"} Nov 04 07:41:46.247016 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.246990 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b" event={"ID":"e8a5b7b3-360d-475c-87b7-6a845394a09c","Type":"ContainerDied","Data":"c1c60ff54ada6bf9df85388beefe3d64de71009b949a312e5be62f7d95f8b03c"} Nov 04 07:41:46.247016 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.247015 5385 scope.go:122] "RemoveContainer" containerID="0d1de9dde96bc674c1447ee6124002858562b9ebf1c29bd9fc42c55b2cc9f476" Nov 04 07:41:46.259979 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.259959 5385 scope.go:122] "RemoveContainer" containerID="4d1b26e1e87ad26f5be3d994366c5f8a1ce778ffc5dbe2df5a0268842ef4c4f8" Nov 04 07:41:46.270387 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.270367 5385 scope.go:122] "RemoveContainer" containerID="0d1de9dde96bc674c1447ee6124002858562b9ebf1c29bd9fc42c55b2cc9f476" Nov 04 07:41:46.270716 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:41:46.270695 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d1de9dde96bc674c1447ee6124002858562b9ebf1c29bd9fc42c55b2cc9f476\": container with ID starting with 0d1de9dde96bc674c1447ee6124002858562b9ebf1c29bd9fc42c55b2cc9f476 not found: ID does not exist" containerID="0d1de9dde96bc674c1447ee6124002858562b9ebf1c29bd9fc42c55b2cc9f476" Nov 04 07:41:46.270780 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.270726 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d1de9dde96bc674c1447ee6124002858562b9ebf1c29bd9fc42c55b2cc9f476"} err="failed to get container status \"0d1de9dde96bc674c1447ee6124002858562b9ebf1c29bd9fc42c55b2cc9f476\": rpc error: code = NotFound desc = could not find container \"0d1de9dde96bc674c1447ee6124002858562b9ebf1c29bd9fc42c55b2cc9f476\": container with ID starting with 0d1de9dde96bc674c1447ee6124002858562b9ebf1c29bd9fc42c55b2cc9f476 not found: ID does not exist" Nov 04 07:41:46.270780 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.270744 5385 scope.go:122] "RemoveContainer" containerID="4d1b26e1e87ad26f5be3d994366c5f8a1ce778ffc5dbe2df5a0268842ef4c4f8" Nov 04 07:41:46.270992 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:41:46.270976 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d1b26e1e87ad26f5be3d994366c5f8a1ce778ffc5dbe2df5a0268842ef4c4f8\": container with ID starting with 4d1b26e1e87ad26f5be3d994366c5f8a1ce778ffc5dbe2df5a0268842ef4c4f8 not found: ID does not exist" containerID="4d1b26e1e87ad26f5be3d994366c5f8a1ce778ffc5dbe2df5a0268842ef4c4f8" Nov 04 07:41:46.271073 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.270996 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d1b26e1e87ad26f5be3d994366c5f8a1ce778ffc5dbe2df5a0268842ef4c4f8"} err="failed to get container status \"4d1b26e1e87ad26f5be3d994366c5f8a1ce778ffc5dbe2df5a0268842ef4c4f8\": rpc error: code = NotFound desc = could not find container \"4d1b26e1e87ad26f5be3d994366c5f8a1ce778ffc5dbe2df5a0268842ef4c4f8\": container with ID starting with 4d1b26e1e87ad26f5be3d994366c5f8a1ce778ffc5dbe2df5a0268842ef4c4f8 not found: ID does not exist" Nov 04 07:41:46.292884 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.292815 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:46.323754 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.323724 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b"] Nov 04 07:41:46.331541 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.331514 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-be2a74-openshift-default-5f4f5978f7-4ht9b"] Nov 04 07:41:46.421285 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.414296 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:46.813440 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.813401 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8a5b7b3-360d-475c-87b7-6a845394a09c" path="/var/lib/kubelet/pods/e8a5b7b3-360d-475c-87b7-6a845394a09c/volumes" Nov 04 07:41:46.891757 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:46.891725 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:47.393563 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:47.393532 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:47.516227 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:47.516196 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-9699b4dc9-g954t"] Nov 04 07:41:47.540976 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:47.540942 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:47.656478 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:47.656381 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:48.297334 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:48.297302 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:49.200738 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:49.200708 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:49.520264 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:49.520177 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:50.179852 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:50.179819 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:50.484462 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:50.484378 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5"] Nov 04 07:41:50.549888 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:50.549849 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:50.649729 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:50.649697 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:50.985049 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:50.985009 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv"] Nov 04 07:41:50.985383 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:50.985314 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" podUID="481004b6-384d-45d9-b7b5-8f79abfd1c06" containerName="maas-api" containerID="cri-o://b2e55daed19627a363df80c7f4ace05c77691dd3f764e3185fc3d712d64a2855" gracePeriod=30 Nov 04 07:41:51.036853 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.036819 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:51.317637 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.317547 5385 generic.go:356] "Generic (PLEG): container finished" podID="481004b6-384d-45d9-b7b5-8f79abfd1c06" containerID="b2e55daed19627a363df80c7f4ace05c77691dd3f764e3185fc3d712d64a2855" exitCode=0 Nov 04 07:41:51.317637 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.317609 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" event={"ID":"481004b6-384d-45d9-b7b5-8f79abfd1c06","Type":"ContainerDied","Data":"b2e55daed19627a363df80c7f4ace05c77691dd3f764e3185fc3d712d64a2855"} Nov 04 07:41:51.347153 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.347132 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:41:51.408986 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.408924 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:51.469717 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.469680 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/481004b6-384d-45d9-b7b5-8f79abfd1c06-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/481004b6-384d-45d9-b7b5-8f79abfd1c06-maas-api-tls\") pod \"481004b6-384d-45d9-b7b5-8f79abfd1c06\" (UID: \"481004b6-384d-45d9-b7b5-8f79abfd1c06\") " Nov 04 07:41:51.470009 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.469728 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/481004b6-384d-45d9-b7b5-8f79abfd1c06-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/481004b6-384d-45d9-b7b5-8f79abfd1c06-openshift-service-ca\") pod \"481004b6-384d-45d9-b7b5-8f79abfd1c06\" (UID: \"481004b6-384d-45d9-b7b5-8f79abfd1c06\") " Nov 04 07:41:51.470009 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.469759 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/481004b6-384d-45d9-b7b5-8f79abfd1c06-metrics-certs\" (UniqueName: \"kubernetes.io/secret/481004b6-384d-45d9-b7b5-8f79abfd1c06-metrics-certs\") pod \"481004b6-384d-45d9-b7b5-8f79abfd1c06\" (UID: \"481004b6-384d-45d9-b7b5-8f79abfd1c06\") " Nov 04 07:41:51.470009 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.469793 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/481004b6-384d-45d9-b7b5-8f79abfd1c06-kube-api-access-cntrt\" (UniqueName: \"kubernetes.io/projected/481004b6-384d-45d9-b7b5-8f79abfd1c06-kube-api-access-cntrt\") pod \"481004b6-384d-45d9-b7b5-8f79abfd1c06\" (UID: \"481004b6-384d-45d9-b7b5-8f79abfd1c06\") " Nov 04 07:41:51.470203 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.470136 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/481004b6-384d-45d9-b7b5-8f79abfd1c06-openshift-service-ca" pod "481004b6-384d-45d9-b7b5-8f79abfd1c06" (UID: "481004b6-384d-45d9-b7b5-8f79abfd1c06"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:41:51.470341 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.470321 5385 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/481004b6-384d-45d9-b7b5-8f79abfd1c06-openshift-service-ca\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:51.472052 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.472021 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/481004b6-384d-45d9-b7b5-8f79abfd1c06-maas-api-tls" pod "481004b6-384d-45d9-b7b5-8f79abfd1c06" (UID: "481004b6-384d-45d9-b7b5-8f79abfd1c06"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:41:51.472164 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.472145 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/481004b6-384d-45d9-b7b5-8f79abfd1c06-kube-api-access-cntrt" pod "481004b6-384d-45d9-b7b5-8f79abfd1c06" (UID: "481004b6-384d-45d9-b7b5-8f79abfd1c06"). InnerVolumeSpecName "kube-api-access-cntrt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:41:51.472261 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.472243 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/481004b6-384d-45d9-b7b5-8f79abfd1c06-metrics-certs" pod "481004b6-384d-45d9-b7b5-8f79abfd1c06" (UID: "481004b6-384d-45d9-b7b5-8f79abfd1c06"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:41:51.517529 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.512404 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:51.571485 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.571401 5385 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/481004b6-384d-45d9-b7b5-8f79abfd1c06-maas-api-tls\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:51.571485 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.571436 5385 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/481004b6-384d-45d9-b7b5-8f79abfd1c06-metrics-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:51.571485 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:51.571448 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cntrt\" (UniqueName: \"kubernetes.io/projected/481004b6-384d-45d9-b7b5-8f79abfd1c06-kube-api-access-cntrt\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:52.198017 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:52.197981 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:52.272120 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:52.271629 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" podUID="5c14cd7f-ad7e-446c-be34-1b251628c970" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Nov 04 07:41:52.328747 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:52.328717 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" Nov 04 07:41:52.328972 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:52.328745 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv" event={"ID":"481004b6-384d-45d9-b7b5-8f79abfd1c06","Type":"ContainerDied","Data":"46dcbe28c25aed98a4e1ddb1b2590583823ca75e5ef257770478f8371f2927f7"} Nov 04 07:41:52.328972 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:52.328814 5385 scope.go:122] "RemoveContainer" containerID="b2e55daed19627a363df80c7f4ace05c77691dd3f764e3185fc3d712d64a2855" Nov 04 07:41:52.357702 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:52.357669 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv"] Nov 04 07:41:52.361881 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:52.361855 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-023307-c4c7c566b-vz5fv"] Nov 04 07:41:52.453263 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:52.453164 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:52.517160 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:52.517115 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" podUID="a112a5b6-387a-48c0-9d0a-d437222311ea" containerName="payload-processing" containerID="cri-o://962c525bce62da85fb259b7a8c7898dd1d46feb834aca7cfb7cf6816836dcc0e" gracePeriod=25 Nov 04 07:41:52.658487 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:52.658454 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:52.822434 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:52.822386 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="481004b6-384d-45d9-b7b5-8f79abfd1c06" path="/var/lib/kubelet/pods/481004b6-384d-45d9-b7b5-8f79abfd1c06/volumes" Nov 04 07:41:53.014133 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.014110 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" Nov 04 07:41:53.036789 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.036528 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:53.101658 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.101623 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c14cd7f-ad7e-446c-be34-1b251628c970-kube-api-access-nrjmp\" (UniqueName: \"kubernetes.io/projected/5c14cd7f-ad7e-446c-be34-1b251628c970-kube-api-access-nrjmp\") pod \"5c14cd7f-ad7e-446c-be34-1b251628c970\" (UID: \"5c14cd7f-ad7e-446c-be34-1b251628c970\") " Nov 04 07:41:53.102043 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.102018 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5c14cd7f-ad7e-446c-be34-1b251628c970-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c14cd7f-ad7e-446c-be34-1b251628c970-plugins-config-volume\") pod \"5c14cd7f-ad7e-446c-be34-1b251628c970\" (UID: \"5c14cd7f-ad7e-446c-be34-1b251628c970\") " Nov 04 07:41:53.102971 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.102932 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c14cd7f-ad7e-446c-be34-1b251628c970-plugins-config-volume" pod "5c14cd7f-ad7e-446c-be34-1b251628c970" (UID: "5c14cd7f-ad7e-446c-be34-1b251628c970"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:41:53.106361 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.106321 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c14cd7f-ad7e-446c-be34-1b251628c970-kube-api-access-nrjmp" pod "5c14cd7f-ad7e-446c-be34-1b251628c970" (UID: "5c14cd7f-ad7e-446c-be34-1b251628c970"). InnerVolumeSpecName "kube-api-access-nrjmp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:41:53.203011 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.202974 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a112a5b6-387a-48c0-9d0a-d437222311ea-kube-api-access-sttpb\" (UniqueName: \"kubernetes.io/projected/a112a5b6-387a-48c0-9d0a-d437222311ea-kube-api-access-sttpb\") pod \"a112a5b6-387a-48c0-9d0a-d437222311ea\" (UID: \"a112a5b6-387a-48c0-9d0a-d437222311ea\") " Nov 04 07:41:53.203215 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.203055 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a112a5b6-387a-48c0-9d0a-d437222311ea-provider-credentials\" (UniqueName: \"kubernetes.io/projected/a112a5b6-387a-48c0-9d0a-d437222311ea-provider-credentials\") pod \"a112a5b6-387a-48c0-9d0a-d437222311ea\" (UID: \"a112a5b6-387a-48c0-9d0a-d437222311ea\") " Nov 04 07:41:53.203215 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.203185 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a112a5b6-387a-48c0-9d0a-d437222311ea-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a112a5b6-387a-48c0-9d0a-d437222311ea-routing-overlay-volume\") pod \"a112a5b6-387a-48c0-9d0a-d437222311ea\" (UID: \"a112a5b6-387a-48c0-9d0a-d437222311ea\") " Nov 04 07:41:53.203215 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.203211 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a112a5b6-387a-48c0-9d0a-d437222311ea-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a112a5b6-387a-48c0-9d0a-d437222311ea-plugins-config-volume\") pod \"a112a5b6-387a-48c0-9d0a-d437222311ea\" (UID: \"a112a5b6-387a-48c0-9d0a-d437222311ea\") " Nov 04 07:41:53.203596 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.203568 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a112a5b6-387a-48c0-9d0a-d437222311ea-routing-overlay-volume" pod "a112a5b6-387a-48c0-9d0a-d437222311ea" (UID: "a112a5b6-387a-48c0-9d0a-d437222311ea"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:41:53.203708 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.203687 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrjmp\" (UniqueName: \"kubernetes.io/projected/5c14cd7f-ad7e-446c-be34-1b251628c970-kube-api-access-nrjmp\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:53.203708 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.203706 5385 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/a112a5b6-387a-48c0-9d0a-d437222311ea-routing-overlay-volume\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:53.203819 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.203716 5385 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5c14cd7f-ad7e-446c-be34-1b251628c970-plugins-config-volume\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:53.203819 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.203713 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a112a5b6-387a-48c0-9d0a-d437222311ea-plugins-config-volume" pod "a112a5b6-387a-48c0-9d0a-d437222311ea" (UID: "a112a5b6-387a-48c0-9d0a-d437222311ea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:41:53.205309 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.205284 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a112a5b6-387a-48c0-9d0a-d437222311ea-kube-api-access-sttpb" pod "a112a5b6-387a-48c0-9d0a-d437222311ea" (UID: "a112a5b6-387a-48c0-9d0a-d437222311ea"). InnerVolumeSpecName "kube-api-access-sttpb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:41:53.205468 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.205444 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a112a5b6-387a-48c0-9d0a-d437222311ea-provider-credentials" pod "a112a5b6-387a-48c0-9d0a-d437222311ea" (UID: "a112a5b6-387a-48c0-9d0a-d437222311ea"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:41:53.304675 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.304576 5385 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/a112a5b6-387a-48c0-9d0a-d437222311ea-provider-credentials\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:53.304675 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.304614 5385 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a112a5b6-387a-48c0-9d0a-d437222311ea-plugins-config-volume\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:53.304675 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.304634 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sttpb\" (UniqueName: \"kubernetes.io/projected/a112a5b6-387a-48c0-9d0a-d437222311ea-kube-api-access-sttpb\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:53.336937 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.336903 5385 generic.go:356] "Generic (PLEG): container finished" podID="5c14cd7f-ad7e-446c-be34-1b251628c970" containerID="f759f2df6e1d3800dbf6af939df5945a3b27ae6fa3d1b64a26790b0be33dfd22" exitCode=0 Nov 04 07:41:53.337198 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.336973 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" Nov 04 07:41:53.337198 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.336988 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" event={"ID":"5c14cd7f-ad7e-446c-be34-1b251628c970","Type":"ContainerDied","Data":"f759f2df6e1d3800dbf6af939df5945a3b27ae6fa3d1b64a26790b0be33dfd22"} Nov 04 07:41:53.337198 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.337057 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt" event={"ID":"5c14cd7f-ad7e-446c-be34-1b251628c970","Type":"ContainerDied","Data":"c86d2da4a6647ae563cd65d19b97f7b8e6535bea26ab698f1271cad2b6181d1a"} Nov 04 07:41:53.337198 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.337091 5385 scope.go:122] "RemoveContainer" containerID="f759f2df6e1d3800dbf6af939df5945a3b27ae6fa3d1b64a26790b0be33dfd22" Nov 04 07:41:53.340636 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.340227 5385 generic.go:356] "Generic (PLEG): container finished" podID="a112a5b6-387a-48c0-9d0a-d437222311ea" containerID="962c525bce62da85fb259b7a8c7898dd1d46feb834aca7cfb7cf6816836dcc0e" exitCode=0 Nov 04 07:41:53.340636 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.340308 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" event={"ID":"a112a5b6-387a-48c0-9d0a-d437222311ea","Type":"ContainerDied","Data":"962c525bce62da85fb259b7a8c7898dd1d46feb834aca7cfb7cf6816836dcc0e"} Nov 04 07:41:53.340636 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.340370 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" event={"ID":"a112a5b6-387a-48c0-9d0a-d437222311ea","Type":"ContainerDied","Data":"b7ede3ea6017a6be234ee44b797f65755f078f9793be3aaea89c7bbe82adb44c"} Nov 04 07:41:53.340636 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.340480 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-9699b4dc9-g954t" Nov 04 07:41:53.356096 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.356068 5385 scope.go:122] "RemoveContainer" containerID="f759f2df6e1d3800dbf6af939df5945a3b27ae6fa3d1b64a26790b0be33dfd22" Nov 04 07:41:53.356574 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:41:53.356548 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f759f2df6e1d3800dbf6af939df5945a3b27ae6fa3d1b64a26790b0be33dfd22\": container with ID starting with f759f2df6e1d3800dbf6af939df5945a3b27ae6fa3d1b64a26790b0be33dfd22 not found: ID does not exist" containerID="f759f2df6e1d3800dbf6af939df5945a3b27ae6fa3d1b64a26790b0be33dfd22" Nov 04 07:41:53.356668 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.356587 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f759f2df6e1d3800dbf6af939df5945a3b27ae6fa3d1b64a26790b0be33dfd22"} err="failed to get container status \"f759f2df6e1d3800dbf6af939df5945a3b27ae6fa3d1b64a26790b0be33dfd22\": rpc error: code = NotFound desc = could not find container \"f759f2df6e1d3800dbf6af939df5945a3b27ae6fa3d1b64a26790b0be33dfd22\": container with ID starting with f759f2df6e1d3800dbf6af939df5945a3b27ae6fa3d1b64a26790b0be33dfd22 not found: ID does not exist" Nov 04 07:41:53.356668 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.356612 5385 scope.go:122] "RemoveContainer" containerID="962c525bce62da85fb259b7a8c7898dd1d46feb834aca7cfb7cf6816836dcc0e" Nov 04 07:41:53.367151 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.367130 5385 scope.go:122] "RemoveContainer" containerID="962c525bce62da85fb259b7a8c7898dd1d46feb834aca7cfb7cf6816836dcc0e" Nov 04 07:41:53.367463 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:41:53.367440 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"962c525bce62da85fb259b7a8c7898dd1d46feb834aca7cfb7cf6816836dcc0e\": container with ID starting with 962c525bce62da85fb259b7a8c7898dd1d46feb834aca7cfb7cf6816836dcc0e not found: ID does not exist" containerID="962c525bce62da85fb259b7a8c7898dd1d46feb834aca7cfb7cf6816836dcc0e" Nov 04 07:41:53.367586 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.367467 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"962c525bce62da85fb259b7a8c7898dd1d46feb834aca7cfb7cf6816836dcc0e"} err="failed to get container status \"962c525bce62da85fb259b7a8c7898dd1d46feb834aca7cfb7cf6816836dcc0e\": rpc error: code = NotFound desc = could not find container \"962c525bce62da85fb259b7a8c7898dd1d46feb834aca7cfb7cf6816836dcc0e\": container with ID starting with 962c525bce62da85fb259b7a8c7898dd1d46feb834aca7cfb7cf6816836dcc0e not found: ID does not exist" Nov 04 07:41:53.375496 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.375454 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt"] Nov 04 07:41:53.380914 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.380881 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-2a8c48-6bc8cb79cb-vxpvt"] Nov 04 07:41:53.390955 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.390931 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-9699b4dc9-g954t"] Nov 04 07:41:53.395329 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.395303 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-9699b4dc9-g954t"] Nov 04 07:41:53.585846 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.585565 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:53.840059 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:53.839961 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:54.587527 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:54.579629 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:54.814272 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:54.814237 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c14cd7f-ad7e-446c-be34-1b251628c970" path="/var/lib/kubelet/pods/5c14cd7f-ad7e-446c-be34-1b251628c970/volumes" Nov 04 07:41:54.814677 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:54.814663 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a112a5b6-387a-48c0-9d0a-d437222311ea" path="/var/lib/kubelet/pods/a112a5b6-387a-48c0-9d0a-d437222311ea/volumes" Nov 04 07:41:55.404881 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:55.404846 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:55.485356 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:55.485317 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" podUID="60ad1363-dd96-42e2-89ae-7c50adfae243" containerName="payload-pre-processing" containerID="cri-o://76e9558329c5d0a2b71c4d3246830945b423f91e3b4fd9d04794242857138e33" gracePeriod=25 Nov 04 07:41:55.832754 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:55.832712 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" Nov 04 07:41:55.955541 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:55.951899 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60ad1363-dd96-42e2-89ae-7c50adfae243-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60ad1363-dd96-42e2-89ae-7c50adfae243-plugins-config-volume\") pod \"60ad1363-dd96-42e2-89ae-7c50adfae243\" (UID: \"60ad1363-dd96-42e2-89ae-7c50adfae243\") " Nov 04 07:41:55.955541 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:55.952198 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60ad1363-dd96-42e2-89ae-7c50adfae243-kube-api-access-s5hfd\" (UniqueName: \"kubernetes.io/projected/60ad1363-dd96-42e2-89ae-7c50adfae243-kube-api-access-s5hfd\") pod \"60ad1363-dd96-42e2-89ae-7c50adfae243\" (UID: \"60ad1363-dd96-42e2-89ae-7c50adfae243\") " Nov 04 07:41:55.972576 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:55.955881 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60ad1363-dd96-42e2-89ae-7c50adfae243-plugins-config-volume" pod "60ad1363-dd96-42e2-89ae-7c50adfae243" (UID: "60ad1363-dd96-42e2-89ae-7c50adfae243"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:41:56.007976 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:56.007717 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60ad1363-dd96-42e2-89ae-7c50adfae243-kube-api-access-s5hfd" pod "60ad1363-dd96-42e2-89ae-7c50adfae243" (UID: "60ad1363-dd96-42e2-89ae-7c50adfae243"). InnerVolumeSpecName "kube-api-access-s5hfd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:41:56.054638 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:56.054555 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5hfd\" (UniqueName: \"kubernetes.io/projected/60ad1363-dd96-42e2-89ae-7c50adfae243-kube-api-access-s5hfd\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:56.054638 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:56.054611 5385 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60ad1363-dd96-42e2-89ae-7c50adfae243-plugins-config-volume\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:56.408552 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:56.399261 5385 generic.go:356] "Generic (PLEG): container finished" podID="60ad1363-dd96-42e2-89ae-7c50adfae243" containerID="76e9558329c5d0a2b71c4d3246830945b423f91e3b4fd9d04794242857138e33" exitCode=0 Nov 04 07:41:56.408552 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:56.399396 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" event={"ID":"60ad1363-dd96-42e2-89ae-7c50adfae243","Type":"ContainerDied","Data":"76e9558329c5d0a2b71c4d3246830945b423f91e3b4fd9d04794242857138e33"} Nov 04 07:41:56.408552 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:56.399453 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" event={"ID":"60ad1363-dd96-42e2-89ae-7c50adfae243","Type":"ContainerDied","Data":"5a75fbdb11e8ca7c30c137e930d83243aa1dc481eccdf8f5f63ad5380a7f2006"} Nov 04 07:41:56.408552 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:56.399487 5385 scope.go:122] "RemoveContainer" containerID="76e9558329c5d0a2b71c4d3246830945b423f91e3b4fd9d04794242857138e33" Nov 04 07:41:56.408552 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:56.399676 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5" Nov 04 07:41:56.535530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:56.508923 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:56.702530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:56.666101 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:56.937603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:56.912655 5385 scope.go:122] "RemoveContainer" containerID="76e9558329c5d0a2b71c4d3246830945b423f91e3b4fd9d04794242857138e33" Nov 04 07:41:57.000812 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:41:57.000774 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76e9558329c5d0a2b71c4d3246830945b423f91e3b4fd9d04794242857138e33\": container with ID starting with 76e9558329c5d0a2b71c4d3246830945b423f91e3b4fd9d04794242857138e33 not found: ID does not exist" containerID="76e9558329c5d0a2b71c4d3246830945b423f91e3b4fd9d04794242857138e33" Nov 04 07:41:57.001136 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:57.001101 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76e9558329c5d0a2b71c4d3246830945b423f91e3b4fd9d04794242857138e33"} err="failed to get container status \"76e9558329c5d0a2b71c4d3246830945b423f91e3b4fd9d04794242857138e33\": rpc error: code = NotFound desc = could not find container \"76e9558329c5d0a2b71c4d3246830945b423f91e3b4fd9d04794242857138e33\": container with ID starting with 76e9558329c5d0a2b71c4d3246830945b423f91e3b4fd9d04794242857138e33 not found: ID does not exist" Nov 04 07:41:57.045627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:57.045585 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:57.173582 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:57.173393 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:57.361607 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:57.349996 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk"] Nov 04 07:41:57.361607 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:57.350247 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" podUID="e3efd517-ed1d-4371-8009-856783d5a8c4" containerName="maas-api" containerID="cri-o://5f2ff44418df210448c75851cb09b0af0ef4791add0def682de86ce204a60cfe" gracePeriod=30 Nov 04 07:41:57.484531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:57.484300 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5"] Nov 04 07:41:57.533530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:57.533369 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-023307-67d89c6dcf-n4md5"] Nov 04 07:41:57.558529 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:57.558335 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" podUID="e3efd517-ed1d-4371-8009-856783d5a8c4" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.74:8443/health\": dial tcp 10.133.0.74:8443: connect: connection refused" Nov 04 07:41:57.912954 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:57.911828 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" podUID="20873ddc-3422-4161-8cba-bb4b55fadc04" containerName="main" probeResult="failure" output="Get \"https://10.133.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 04 07:41:58.181680 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:58.181582 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" podUID="c2b40c45-d563-4994-876c-b150d0403dd6" containerName="main" probeResult="failure" output="Get \"https://10.133.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 04 07:41:58.386793 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:58.384044 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" podUID="2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 04 07:41:58.465582 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:58.463205 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:58.635552 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:58.611779 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:58.691538 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:58.691437 5385 generic.go:356] "Generic (PLEG): container finished" podID="e3efd517-ed1d-4371-8009-856783d5a8c4" containerID="5f2ff44418df210448c75851cb09b0af0ef4791add0def682de86ce204a60cfe" exitCode=0 Nov 04 07:41:58.691538 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:58.691500 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" event={"ID":"e3efd517-ed1d-4371-8009-856783d5a8c4","Type":"ContainerDied","Data":"5f2ff44418df210448c75851cb09b0af0ef4791add0def682de86ce204a60cfe"} Nov 04 07:41:58.779576 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:58.776824 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:41:58.912552 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:58.900517 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e3efd517-ed1d-4371-8009-856783d5a8c4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3efd517-ed1d-4371-8009-856783d5a8c4-maas-api-tls\") pod \"e3efd517-ed1d-4371-8009-856783d5a8c4\" (UID: \"e3efd517-ed1d-4371-8009-856783d5a8c4\") " Nov 04 07:41:58.966530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:58.913381 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e3efd517-ed1d-4371-8009-856783d5a8c4-kube-api-access-zvtbk\" (UniqueName: \"kubernetes.io/projected/e3efd517-ed1d-4371-8009-856783d5a8c4-kube-api-access-zvtbk\") pod \"e3efd517-ed1d-4371-8009-856783d5a8c4\" (UID: \"e3efd517-ed1d-4371-8009-856783d5a8c4\") " Nov 04 07:41:58.967328 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:58.962252 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60ad1363-dd96-42e2-89ae-7c50adfae243" path="/var/lib/kubelet/pods/60ad1363-dd96-42e2-89ae-7c50adfae243/volumes" Nov 04 07:41:58.969946 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:58.969923 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e3efd517-ed1d-4371-8009-856783d5a8c4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3efd517-ed1d-4371-8009-856783d5a8c4-metrics-certs\") pod \"e3efd517-ed1d-4371-8009-856783d5a8c4\" (UID: \"e3efd517-ed1d-4371-8009-856783d5a8c4\") " Nov 04 07:41:58.978422 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:58.970486 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e3efd517-ed1d-4371-8009-856783d5a8c4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3efd517-ed1d-4371-8009-856783d5a8c4-openshift-service-ca\") pod \"e3efd517-ed1d-4371-8009-856783d5a8c4\" (UID: \"e3efd517-ed1d-4371-8009-856783d5a8c4\") " Nov 04 07:41:58.986754 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:58.979714 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3efd517-ed1d-4371-8009-856783d5a8c4-openshift-service-ca" pod "e3efd517-ed1d-4371-8009-856783d5a8c4" (UID: "e3efd517-ed1d-4371-8009-856783d5a8c4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:41:59.018267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.018229 5385 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-74854d8ccf-lhm7v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 07:41:59.018561 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.018530 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-74854d8ccf-lhm7v" podUID="ae22c880-563f-4c2c-9461-b1e185adf358" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 07:41:59.020806 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.020755 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3efd517-ed1d-4371-8009-856783d5a8c4-kube-api-access-zvtbk" pod "e3efd517-ed1d-4371-8009-856783d5a8c4" (UID: "e3efd517-ed1d-4371-8009-856783d5a8c4"). InnerVolumeSpecName "kube-api-access-zvtbk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:41:59.100530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.075069 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:59.100530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.076970 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3efd517-ed1d-4371-8009-856783d5a8c4-maas-api-tls" pod "e3efd517-ed1d-4371-8009-856783d5a8c4" (UID: "e3efd517-ed1d-4371-8009-856783d5a8c4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:41:59.100530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.094849 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvtbk\" (UniqueName: \"kubernetes.io/projected/e3efd517-ed1d-4371-8009-856783d5a8c4-kube-api-access-zvtbk\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:59.100530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.094883 5385 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3efd517-ed1d-4371-8009-856783d5a8c4-openshift-service-ca\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:59.100530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.094900 5385 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e3efd517-ed1d-4371-8009-856783d5a8c4-maas-api-tls\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:59.135530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.132771 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3efd517-ed1d-4371-8009-856783d5a8c4-metrics-certs" pod "e3efd517-ed1d-4371-8009-856783d5a8c4" (UID: "e3efd517-ed1d-4371-8009-856783d5a8c4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:41:59.217540 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.203028 5385 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3efd517-ed1d-4371-8009-856783d5a8c4-metrics-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:41:59.218576 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.218496 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:59.526013 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.525980 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:41:59.701427 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.701390 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" event={"ID":"e3efd517-ed1d-4371-8009-856783d5a8c4","Type":"ContainerDied","Data":"c75e41cae89a8662ffc2b7c2b723e177f14d4dea1fd1143acdb40825edf86cc3"} Nov 04 07:41:59.701923 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.701452 5385 scope.go:122] "RemoveContainer" containerID="5f2ff44418df210448c75851cb09b0af0ef4791add0def682de86ce204a60cfe" Nov 04 07:41:59.701923 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.701600 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk" Nov 04 07:41:59.725420 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.725394 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk"] Nov 04 07:41:59.730206 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.730185 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-5c1e58-84479b768c-gl9nk"] Nov 04 07:41:59.995595 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:41:59.995563 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:00.527237 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:00.527203 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:00.812164 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:00.812084 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e3efd517-ed1d-4371-8009-856783d5a8c4" path="/var/lib/kubelet/pods/e3efd517-ed1d-4371-8009-856783d5a8c4/volumes" Nov 04 07:42:00.876296 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:00.876265 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:42:01.257171 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.257132 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" podUID="c2b40c45-d563-4994-876c-b150d0403dd6" containerName="main" containerID="cri-o://600e32d5b2ee156ba580e749e9703769dee45938a0e0e4be8450150c5675e24a" gracePeriod=45 Nov 04 07:42:01.267006 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.266972 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" podUID="20873ddc-3422-4161-8cba-bb4b55fadc04" containerName="main" containerID="cri-o://da21057a57bf2340d516e4f80856fb1ac06c477744aa9c4c9fc1f3e5a06e7cae" gracePeriod=45 Nov 04 07:42:01.271189 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.271160 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" podUID="2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff" containerName="main" containerID="cri-o://0fd3995a54be0922c32b328411ce66bfcb4be5858534f59d14ad3f8e673b003e" gracePeriod=45 Nov 04 07:42:01.311367 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.311338 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:01.375122 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.374832 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" podUID="2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff" containerName="main" probeResult="failure" output="Get \"https://10.133.0.51:8000/ready\": dial tcp 10.133.0.51:8000: connect: connection refused" Nov 04 07:42:01.556199 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.556175 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:42:01.573863 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.573838 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:42:01.578906 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.578879 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:42:01.661737 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.661703 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-model-cache\") pod \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " Nov 04 07:42:01.661933 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.661748 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20873ddc-3422-4161-8cba-bb4b55fadc04-kube-api-access-j6p4t\" (UniqueName: \"kubernetes.io/projected/20873ddc-3422-4161-8cba-bb4b55fadc04-kube-api-access-j6p4t\") pod \"20873ddc-3422-4161-8cba-bb4b55fadc04\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " Nov 04 07:42:01.661933 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.661771 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-dshm\") pod \"c2b40c45-d563-4994-876c-b150d0403dd6\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " Nov 04 07:42:01.661933 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.661809 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-model-cache\") pod \"20873ddc-3422-4161-8cba-bb4b55fadc04\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " Nov 04 07:42:01.661933 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.661853 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-tmp-dir\") pod \"c2b40c45-d563-4994-876c-b150d0403dd6\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " Nov 04 07:42:01.661933 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.661882 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-home\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-home\") pod \"20873ddc-3422-4161-8cba-bb4b55fadc04\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " Nov 04 07:42:01.661933 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.661915 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-tmp-dir\") pod \"20873ddc-3422-4161-8cba-bb4b55fadc04\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " Nov 04 07:42:01.662297 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.661943 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c2b40c45-d563-4994-876c-b150d0403dd6-tls-certs\" (UniqueName: \"kubernetes.io/secret/c2b40c45-d563-4994-876c-b150d0403dd6-tls-certs\") pod \"c2b40c45-d563-4994-876c-b150d0403dd6\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " Nov 04 07:42:01.662297 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.661984 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c2b40c45-d563-4994-876c-b150d0403dd6-kube-api-access-t7vhm\" (UniqueName: \"kubernetes.io/projected/c2b40c45-d563-4994-876c-b150d0403dd6-kube-api-access-t7vhm\") pod \"c2b40c45-d563-4994-876c-b150d0403dd6\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " Nov 04 07:42:01.662297 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.662013 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-dshm\") pod \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " Nov 04 07:42:01.662297 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.662011 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-model-cache" pod "2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff" (UID: "2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:01.662297 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.662049 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-tls-certs\" (UniqueName: \"kubernetes.io/secret/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-tls-certs\") pod \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " Nov 04 07:42:01.662297 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.662075 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-dshm\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-dshm\") pod \"20873ddc-3422-4161-8cba-bb4b55fadc04\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " Nov 04 07:42:01.662297 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.662100 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-model-cache\") pod \"c2b40c45-d563-4994-876c-b150d0403dd6\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " Nov 04 07:42:01.662297 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.662141 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-home\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-home\") pod \"c2b40c45-d563-4994-876c-b150d0403dd6\" (UID: \"c2b40c45-d563-4994-876c-b150d0403dd6\") " Nov 04 07:42:01.662297 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.662149 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-model-cache" pod "20873ddc-3422-4161-8cba-bb4b55fadc04" (UID: "20873ddc-3422-4161-8cba-bb4b55fadc04"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:01.662297 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.662191 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/20873ddc-3422-4161-8cba-bb4b55fadc04-tls-certs\" (UniqueName: \"kubernetes.io/secret/20873ddc-3422-4161-8cba-bb4b55fadc04-tls-certs\") pod \"20873ddc-3422-4161-8cba-bb4b55fadc04\" (UID: \"20873ddc-3422-4161-8cba-bb4b55fadc04\") " Nov 04 07:42:01.662297 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.662247 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-tmp-dir" pod "20873ddc-3422-4161-8cba-bb4b55fadc04" (UID: "20873ddc-3422-4161-8cba-bb4b55fadc04"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:01.662297 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.662286 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-kube-api-access-m59w2\" (UniqueName: \"kubernetes.io/projected/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-kube-api-access-m59w2\") pod \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " Nov 04 07:42:01.662943 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.662314 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-tmp-dir\") pod \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " Nov 04 07:42:01.662943 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.662346 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-home" pod "20873ddc-3422-4161-8cba-bb4b55fadc04" (UID: "20873ddc-3422-4161-8cba-bb4b55fadc04"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:01.662943 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.662354 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-home\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-home\") pod \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\" (UID: \"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff\") " Nov 04 07:42:01.662943 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.662820 5385 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-model-cache\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.662943 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.662841 5385 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-model-cache\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.662943 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.662855 5385 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-home\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.662943 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.662868 5385 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-tmp-dir\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.663316 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.663055 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-home" pod "2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff" (UID: "2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:01.663316 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.663176 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-tmp-dir" pod "c2b40c45-d563-4994-876c-b150d0403dd6" (UID: "c2b40c45-d563-4994-876c-b150d0403dd6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:01.663316 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.663251 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-tmp-dir" pod "2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff" (UID: "2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:01.663316 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.663311 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-model-cache" pod "c2b40c45-d563-4994-876c-b150d0403dd6" (UID: "c2b40c45-d563-4994-876c-b150d0403dd6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:01.663558 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.663368 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-home" pod "c2b40c45-d563-4994-876c-b150d0403dd6" (UID: "c2b40c45-d563-4994-876c-b150d0403dd6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:01.665290 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.665163 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20873ddc-3422-4161-8cba-bb4b55fadc04-kube-api-access-j6p4t" pod "20873ddc-3422-4161-8cba-bb4b55fadc04" (UID: "20873ddc-3422-4161-8cba-bb4b55fadc04"). InnerVolumeSpecName "kube-api-access-j6p4t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:01.665290 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.665243 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-tls-certs" pod "2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff" (UID: "2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:42:01.666297 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.666262 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-dshm" pod "2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff" (UID: "2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:01.666570 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.666537 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-kube-api-access-m59w2" pod "2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff" (UID: "2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff"). InnerVolumeSpecName "kube-api-access-m59w2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:01.666570 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.666553 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2b40c45-d563-4994-876c-b150d0403dd6-tls-certs" pod "c2b40c45-d563-4994-876c-b150d0403dd6" (UID: "c2b40c45-d563-4994-876c-b150d0403dd6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:42:01.666746 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.666575 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2b40c45-d563-4994-876c-b150d0403dd6-kube-api-access-t7vhm" pod "c2b40c45-d563-4994-876c-b150d0403dd6" (UID: "c2b40c45-d563-4994-876c-b150d0403dd6"). InnerVolumeSpecName "kube-api-access-t7vhm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:01.666746 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.666709 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-dshm" pod "20873ddc-3422-4161-8cba-bb4b55fadc04" (UID: "20873ddc-3422-4161-8cba-bb4b55fadc04"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:01.666895 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.666872 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-dshm" pod "c2b40c45-d563-4994-876c-b150d0403dd6" (UID: "c2b40c45-d563-4994-876c-b150d0403dd6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:01.666982 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.666966 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20873ddc-3422-4161-8cba-bb4b55fadc04-tls-certs" pod "20873ddc-3422-4161-8cba-bb4b55fadc04" (UID: "20873ddc-3422-4161-8cba-bb4b55fadc04"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:42:01.718620 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.718583 5385 generic.go:356] "Generic (PLEG): container finished" podID="2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff" containerID="0fd3995a54be0922c32b328411ce66bfcb4be5858534f59d14ad3f8e673b003e" exitCode=0 Nov 04 07:42:01.718812 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.718629 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" Nov 04 07:42:01.718812 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.718666 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" event={"ID":"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff","Type":"ContainerDied","Data":"0fd3995a54be0922c32b328411ce66bfcb4be5858534f59d14ad3f8e673b003e"} Nov 04 07:42:01.718812 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.718714 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l" event={"ID":"2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff","Type":"ContainerDied","Data":"98e8432c24d7b1c7875d1393feb265fd439f4873c49a4b135b858131af9ad760"} Nov 04 07:42:01.718812 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.718735 5385 scope.go:122] "RemoveContainer" containerID="0fd3995a54be0922c32b328411ce66bfcb4be5858534f59d14ad3f8e673b003e" Nov 04 07:42:01.720333 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.720218 5385 generic.go:356] "Generic (PLEG): container finished" podID="c2b40c45-d563-4994-876c-b150d0403dd6" containerID="600e32d5b2ee156ba580e749e9703769dee45938a0e0e4be8450150c5675e24a" exitCode=0 Nov 04 07:42:01.720333 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.720246 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" event={"ID":"c2b40c45-d563-4994-876c-b150d0403dd6","Type":"ContainerDied","Data":"600e32d5b2ee156ba580e749e9703769dee45938a0e0e4be8450150c5675e24a"} Nov 04 07:42:01.720333 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.720278 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" Nov 04 07:42:01.720608 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.720285 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g" event={"ID":"c2b40c45-d563-4994-876c-b150d0403dd6","Type":"ContainerDied","Data":"201c6fea2a61244d79d629e79f263c9609fc2d5fd4a2a7f8027e09d805dafff1"} Nov 04 07:42:01.721806 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.721784 5385 generic.go:356] "Generic (PLEG): container finished" podID="20873ddc-3422-4161-8cba-bb4b55fadc04" containerID="da21057a57bf2340d516e4f80856fb1ac06c477744aa9c4c9fc1f3e5a06e7cae" exitCode=0 Nov 04 07:42:01.721944 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.721817 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" Nov 04 07:42:01.721944 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.721840 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" event={"ID":"20873ddc-3422-4161-8cba-bb4b55fadc04","Type":"ContainerDied","Data":"da21057a57bf2340d516e4f80856fb1ac06c477744aa9c4c9fc1f3e5a06e7cae"} Nov 04 07:42:01.721944 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.721863 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n" event={"ID":"20873ddc-3422-4161-8cba-bb4b55fadc04","Type":"ContainerDied","Data":"316386ebb5a20a39442377bf948cc02e5fa28249d372fc5ac3c60962c8112ff1"} Nov 04 07:42:01.729556 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.729533 5385 scope.go:122] "RemoveContainer" containerID="0fd3995a54be0922c32b328411ce66bfcb4be5858534f59d14ad3f8e673b003e" Nov 04 07:42:01.729919 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:01.729900 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fd3995a54be0922c32b328411ce66bfcb4be5858534f59d14ad3f8e673b003e\": container with ID starting with 0fd3995a54be0922c32b328411ce66bfcb4be5858534f59d14ad3f8e673b003e not found: ID does not exist" containerID="0fd3995a54be0922c32b328411ce66bfcb4be5858534f59d14ad3f8e673b003e" Nov 04 07:42:01.729997 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.729926 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fd3995a54be0922c32b328411ce66bfcb4be5858534f59d14ad3f8e673b003e"} err="failed to get container status \"0fd3995a54be0922c32b328411ce66bfcb4be5858534f59d14ad3f8e673b003e\": rpc error: code = NotFound desc = could not find container \"0fd3995a54be0922c32b328411ce66bfcb4be5858534f59d14ad3f8e673b003e\": container with ID starting with 0fd3995a54be0922c32b328411ce66bfcb4be5858534f59d14ad3f8e673b003e not found: ID does not exist" Nov 04 07:42:01.729997 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.729943 5385 scope.go:122] "RemoveContainer" containerID="600e32d5b2ee156ba580e749e9703769dee45938a0e0e4be8450150c5675e24a" Nov 04 07:42:01.752529 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.752493 5385 scope.go:122] "RemoveContainer" containerID="600e32d5b2ee156ba580e749e9703769dee45938a0e0e4be8450150c5675e24a" Nov 04 07:42:01.752843 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:01.752822 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"600e32d5b2ee156ba580e749e9703769dee45938a0e0e4be8450150c5675e24a\": container with ID starting with 600e32d5b2ee156ba580e749e9703769dee45938a0e0e4be8450150c5675e24a not found: ID does not exist" containerID="600e32d5b2ee156ba580e749e9703769dee45938a0e0e4be8450150c5675e24a" Nov 04 07:42:01.752903 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.752853 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"600e32d5b2ee156ba580e749e9703769dee45938a0e0e4be8450150c5675e24a"} err="failed to get container status \"600e32d5b2ee156ba580e749e9703769dee45938a0e0e4be8450150c5675e24a\": rpc error: code = NotFound desc = could not find container \"600e32d5b2ee156ba580e749e9703769dee45938a0e0e4be8450150c5675e24a\": container with ID starting with 600e32d5b2ee156ba580e749e9703769dee45938a0e0e4be8450150c5675e24a not found: ID does not exist" Nov 04 07:42:01.752903 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.752872 5385 scope.go:122] "RemoveContainer" containerID="da21057a57bf2340d516e4f80856fb1ac06c477744aa9c4c9fc1f3e5a06e7cae" Nov 04 07:42:01.762321 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.762276 5385 scope.go:122] "RemoveContainer" containerID="da21057a57bf2340d516e4f80856fb1ac06c477744aa9c4c9fc1f3e5a06e7cae" Nov 04 07:42:01.762792 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:01.762766 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da21057a57bf2340d516e4f80856fb1ac06c477744aa9c4c9fc1f3e5a06e7cae\": container with ID starting with da21057a57bf2340d516e4f80856fb1ac06c477744aa9c4c9fc1f3e5a06e7cae not found: ID does not exist" containerID="da21057a57bf2340d516e4f80856fb1ac06c477744aa9c4c9fc1f3e5a06e7cae" Nov 04 07:42:01.762891 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.762796 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da21057a57bf2340d516e4f80856fb1ac06c477744aa9c4c9fc1f3e5a06e7cae"} err="failed to get container status \"da21057a57bf2340d516e4f80856fb1ac06c477744aa9c4c9fc1f3e5a06e7cae\": rpc error: code = NotFound desc = could not find container \"da21057a57bf2340d516e4f80856fb1ac06c477744aa9c4c9fc1f3e5a06e7cae\": container with ID starting with da21057a57bf2340d516e4f80856fb1ac06c477744aa9c4c9fc1f3e5a06e7cae not found: ID does not exist" Nov 04 07:42:01.763581 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.763557 5385 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-tmp-dir\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.763581 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.763583 5385 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c2b40c45-d563-4994-876c-b150d0403dd6-tls-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.763716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.763593 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7vhm\" (UniqueName: \"kubernetes.io/projected/c2b40c45-d563-4994-876c-b150d0403dd6-kube-api-access-t7vhm\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.763716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.763603 5385 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-dshm\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.763716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.763613 5385 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-tls-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.763716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.763621 5385 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/20873ddc-3422-4161-8cba-bb4b55fadc04-dshm\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.763716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.763629 5385 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-model-cache\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.763716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.763638 5385 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-home\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.763716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.763647 5385 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/20873ddc-3422-4161-8cba-bb4b55fadc04-tls-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.763716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.763654 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m59w2\" (UniqueName: \"kubernetes.io/projected/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-kube-api-access-m59w2\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.763716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.763662 5385 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-tmp-dir\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.763716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.763670 5385 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff-home\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.763716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.763702 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6p4t\" (UniqueName: \"kubernetes.io/projected/20873ddc-3422-4161-8cba-bb4b55fadc04-kube-api-access-j6p4t\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.763716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.763711 5385 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c2b40c45-d563-4994-876c-b150d0403dd6-dshm\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:01.765079 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.765059 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l"] Nov 04 07:42:01.771359 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.771335 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/premium-w2-016e6c-kserve-66569fc68c-cxg2l"] Nov 04 07:42:01.783167 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.783108 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g"] Nov 04 07:42:01.787316 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.787296 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/distinct-2-w2-016e6c-kserve-5cf5d4b445-nj96g"] Nov 04 07:42:01.799078 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.799059 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n"] Nov 04 07:42:01.803865 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:01.803840 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-016e6c/distinct-w2-016e6c-kserve-5dfdf5f774-c2s9n"] Nov 04 07:42:02.271155 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:02.271124 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:02.634483 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:02.634434 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:02.812347 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:02.812316 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20873ddc-3422-4161-8cba-bb4b55fadc04" path="/var/lib/kubelet/pods/20873ddc-3422-4161-8cba-bb4b55fadc04/volumes" Nov 04 07:42:02.812737 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:02.812723 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff" path="/var/lib/kubelet/pods/2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff/volumes" Nov 04 07:42:02.813103 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:02.813091 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c2b40c45-d563-4994-876c-b150d0403dd6" path="/var/lib/kubelet/pods/c2b40c45-d563-4994-876c-b150d0403dd6/volumes" Nov 04 07:42:03.483632 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:03.483595 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:03.953650 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:03.953611 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:04.286965 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:04.286882 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:05.052379 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:05.052348 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:05.169124 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:05.169076 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:05.324188 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:05.324111 5385 scope.go:122] "RemoveContainer" containerID="eebcb0d628d1dcab4f6f6c9d75597c9ea7361a8bf0e638cf610f53c36d6ac64a" Nov 04 07:42:05.554921 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:05.554887 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:05.964451 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:05.964416 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:06.775522 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:06.775475 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:07.676794 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:07.676762 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:08.475095 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:08.475058 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:08.910173 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:08.910139 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:09.346261 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:09.346157 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:10.759067 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:10.759028 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:11.596530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:11.593986 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:11.722997 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:11.722959 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:11.969677 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:11.969636 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 07:42:11.970347 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:11.970294 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" podUID="99d41a69-904b-44b9-9a38-253d859dac34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 07:42:12.175215 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:12.175178 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:12.334580 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:12.321182 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:12.553638 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:12.553606 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:12.653691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:12.653657 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:12.919101 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:12.919013 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:13.296749 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.296662 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:13.426477 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.426438 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf"] Nov 04 07:42:13.426800 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.426762 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" podUID="a52cfeca-0537-4751-8db1-8e1da56f1968" containerName="maas-api" containerID="cri-o://5893060fe04e2e6aa5a4edc711c498b3077b1e3199d11693cb4068e1e446c14b" gracePeriod=30 Nov 04 07:42:13.465295 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.465254 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:13.731275 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.731203 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:13.801791 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.801765 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:42:13.823706 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.823672 5385 generic.go:356] "Generic (PLEG): container finished" podID="a52cfeca-0537-4751-8db1-8e1da56f1968" containerID="5893060fe04e2e6aa5a4edc711c498b3077b1e3199d11693cb4068e1e446c14b" exitCode=0 Nov 04 07:42:13.823905 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.823716 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" Nov 04 07:42:13.823905 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.823732 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" event={"ID":"a52cfeca-0537-4751-8db1-8e1da56f1968","Type":"ContainerDied","Data":"5893060fe04e2e6aa5a4edc711c498b3077b1e3199d11693cb4068e1e446c14b"} Nov 04 07:42:13.823905 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.823787 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf" event={"ID":"a52cfeca-0537-4751-8db1-8e1da56f1968","Type":"ContainerDied","Data":"d3f6212c2e44c6d570fb98e16d8856c88fb368f0f0c930b7d9f91b0c24060de6"} Nov 04 07:42:13.823905 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.823819 5385 scope.go:122] "RemoveContainer" containerID="5893060fe04e2e6aa5a4edc711c498b3077b1e3199d11693cb4068e1e446c14b" Nov 04 07:42:13.835434 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.835410 5385 scope.go:122] "RemoveContainer" containerID="5893060fe04e2e6aa5a4edc711c498b3077b1e3199d11693cb4068e1e446c14b" Nov 04 07:42:13.835850 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:13.835824 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5893060fe04e2e6aa5a4edc711c498b3077b1e3199d11693cb4068e1e446c14b\": container with ID starting with 5893060fe04e2e6aa5a4edc711c498b3077b1e3199d11693cb4068e1e446c14b not found: ID does not exist" containerID="5893060fe04e2e6aa5a4edc711c498b3077b1e3199d11693cb4068e1e446c14b" Nov 04 07:42:13.835946 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.835862 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5893060fe04e2e6aa5a4edc711c498b3077b1e3199d11693cb4068e1e446c14b"} err="failed to get container status \"5893060fe04e2e6aa5a4edc711c498b3077b1e3199d11693cb4068e1e446c14b\": rpc error: code = NotFound desc = could not find container \"5893060fe04e2e6aa5a4edc711c498b3077b1e3199d11693cb4068e1e446c14b\": container with ID starting with 5893060fe04e2e6aa5a4edc711c498b3077b1e3199d11693cb4068e1e446c14b not found: ID does not exist" Nov 04 07:42:13.848057 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.848032 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:13.912069 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.912036 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a52cfeca-0537-4751-8db1-8e1da56f1968-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a52cfeca-0537-4751-8db1-8e1da56f1968-maas-api-tls\") pod \"a52cfeca-0537-4751-8db1-8e1da56f1968\" (UID: \"a52cfeca-0537-4751-8db1-8e1da56f1968\") " Nov 04 07:42:13.912069 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.912074 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a52cfeca-0537-4751-8db1-8e1da56f1968-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a52cfeca-0537-4751-8db1-8e1da56f1968-metrics-certs\") pod \"a52cfeca-0537-4751-8db1-8e1da56f1968\" (UID: \"a52cfeca-0537-4751-8db1-8e1da56f1968\") " Nov 04 07:42:13.912347 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.912191 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a52cfeca-0537-4751-8db1-8e1da56f1968-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a52cfeca-0537-4751-8db1-8e1da56f1968-openshift-service-ca\") pod \"a52cfeca-0537-4751-8db1-8e1da56f1968\" (UID: \"a52cfeca-0537-4751-8db1-8e1da56f1968\") " Nov 04 07:42:13.912347 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.912266 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a52cfeca-0537-4751-8db1-8e1da56f1968-kube-api-access-5kwc6\" (UniqueName: \"kubernetes.io/projected/a52cfeca-0537-4751-8db1-8e1da56f1968-kube-api-access-5kwc6\") pod \"a52cfeca-0537-4751-8db1-8e1da56f1968\" (UID: \"a52cfeca-0537-4751-8db1-8e1da56f1968\") " Nov 04 07:42:13.912816 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.912785 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a52cfeca-0537-4751-8db1-8e1da56f1968-openshift-service-ca" pod "a52cfeca-0537-4751-8db1-8e1da56f1968" (UID: "a52cfeca-0537-4751-8db1-8e1da56f1968"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:42:13.914820 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.914792 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a52cfeca-0537-4751-8db1-8e1da56f1968-metrics-certs" pod "a52cfeca-0537-4751-8db1-8e1da56f1968" (UID: "a52cfeca-0537-4751-8db1-8e1da56f1968"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:42:13.914929 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.914910 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a52cfeca-0537-4751-8db1-8e1da56f1968-maas-api-tls" pod "a52cfeca-0537-4751-8db1-8e1da56f1968" (UID: "a52cfeca-0537-4751-8db1-8e1da56f1968"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:42:13.915033 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:13.915011 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a52cfeca-0537-4751-8db1-8e1da56f1968-kube-api-access-5kwc6" pod "a52cfeca-0537-4751-8db1-8e1da56f1968" (UID: "a52cfeca-0537-4751-8db1-8e1da56f1968"). InnerVolumeSpecName "kube-api-access-5kwc6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:14.013324 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.013282 5385 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a52cfeca-0537-4751-8db1-8e1da56f1968-maas-api-tls\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:14.013324 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.013318 5385 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a52cfeca-0537-4751-8db1-8e1da56f1968-metrics-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:14.013324 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.013328 5385 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a52cfeca-0537-4751-8db1-8e1da56f1968-openshift-service-ca\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:14.013604 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.013338 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kwc6\" (UniqueName: \"kubernetes.io/projected/a52cfeca-0537-4751-8db1-8e1da56f1968-kube-api-access-5kwc6\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:14.156615 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.156238 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf"] Nov 04 07:42:14.158458 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.158429 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-016e6c-78f9bcffc8-jvbbf"] Nov 04 07:42:14.792445 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.792413 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:14.813960 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.813933 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a52cfeca-0537-4751-8db1-8e1da56f1968" path="/var/lib/kubelet/pods/a52cfeca-0537-4751-8db1-8e1da56f1968/volumes" Nov 04 07:42:14.915243 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915206 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s"] Nov 04 07:42:14.915763 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915743 5385 cpu_manager.go:414] "containerMap: removing container" podUID="a52cfeca-0537-4751-8db1-8e1da56f1968" containerName="maas-api" Nov 04 07:42:14.915763 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915764 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a52cfeca-0537-4751-8db1-8e1da56f1968" containerName="maas-api" Nov 04 07:42:14.915940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915782 5385 cpu_manager.go:414] "containerMap: removing container" podUID="5c14cd7f-ad7e-446c-be34-1b251628c970" containerName="payload-processing" Nov 04 07:42:14.915940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915789 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c14cd7f-ad7e-446c-be34-1b251628c970" containerName="payload-processing" Nov 04 07:42:14.915940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915802 5385 cpu_manager.go:414] "containerMap: removing container" podUID="20873ddc-3422-4161-8cba-bb4b55fadc04" containerName="main" Nov 04 07:42:14.915940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915809 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20873ddc-3422-4161-8cba-bb4b55fadc04" containerName="main" Nov 04 07:42:14.915940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915824 5385 cpu_manager.go:414] "containerMap: removing container" podUID="481004b6-384d-45d9-b7b5-8f79abfd1c06" containerName="maas-api" Nov 04 07:42:14.915940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915832 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="481004b6-384d-45d9-b7b5-8f79abfd1c06" containerName="maas-api" Nov 04 07:42:14.915940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915846 5385 cpu_manager.go:414] "containerMap: removing container" podUID="2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff" containerName="main" Nov 04 07:42:14.915940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915853 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff" containerName="main" Nov 04 07:42:14.915940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915869 5385 cpu_manager.go:414] "containerMap: removing container" podUID="34fb6ab3-7f51-47c3-8d8d-dd057d054eb0" containerName="maas-api" Nov 04 07:42:14.915940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915876 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34fb6ab3-7f51-47c3-8d8d-dd057d054eb0" containerName="maas-api" Nov 04 07:42:14.915940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915889 5385 cpu_manager.go:414] "containerMap: removing container" podUID="c2b40c45-d563-4994-876c-b150d0403dd6" containerName="main" Nov 04 07:42:14.915940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915896 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c2b40c45-d563-4994-876c-b150d0403dd6" containerName="main" Nov 04 07:42:14.915940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915910 5385 cpu_manager.go:414] "containerMap: removing container" podUID="a112a5b6-387a-48c0-9d0a-d437222311ea" containerName="payload-processing" Nov 04 07:42:14.915940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915917 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a112a5b6-387a-48c0-9d0a-d437222311ea" containerName="payload-processing" Nov 04 07:42:14.915940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915932 5385 cpu_manager.go:414] "containerMap: removing container" podUID="5db0f011-9979-4ecd-9525-5a02cfd6c8c8" containerName="istio-proxy" Nov 04 07:42:14.915940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915940 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5db0f011-9979-4ecd-9525-5a02cfd6c8c8" containerName="istio-proxy" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915962 5385 cpu_manager.go:414] "containerMap: removing container" podUID="e8a5b7b3-360d-475c-87b7-6a845394a09c" containerName="istio-proxy" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915970 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8a5b7b3-360d-475c-87b7-6a845394a09c" containerName="istio-proxy" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915979 5385 cpu_manager.go:414] "containerMap: removing container" podUID="e3efd517-ed1d-4371-8009-856783d5a8c4" containerName="maas-api" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915986 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e3efd517-ed1d-4371-8009-856783d5a8c4" containerName="maas-api" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.915998 5385 cpu_manager.go:414] "containerMap: removing container" podUID="e8a5b7b3-360d-475c-87b7-6a845394a09c" containerName="istio-proxy" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.916005 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8a5b7b3-360d-475c-87b7-6a845394a09c" containerName="istio-proxy" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.916019 5385 cpu_manager.go:414] "containerMap: removing container" podUID="60ad1363-dd96-42e2-89ae-7c50adfae243" containerName="payload-pre-processing" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.916026 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60ad1363-dd96-42e2-89ae-7c50adfae243" containerName="payload-pre-processing" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.916133 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="a52cfeca-0537-4751-8db1-8e1da56f1968" containerName="maas-api" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.916145 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c14cd7f-ad7e-446c-be34-1b251628c970" containerName="payload-processing" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.916155 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="20873ddc-3422-4161-8cba-bb4b55fadc04" containerName="main" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.916165 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="481004b6-384d-45d9-b7b5-8f79abfd1c06" containerName="maas-api" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.916174 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="2411a0ad-44d7-4ff2-9cdd-c53f0f52f2ff" containerName="main" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.916183 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="34fb6ab3-7f51-47c3-8d8d-dd057d054eb0" containerName="maas-api" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.916196 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="c2b40c45-d563-4994-876c-b150d0403dd6" containerName="main" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.916205 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="a112a5b6-387a-48c0-9d0a-d437222311ea" containerName="payload-processing" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.916215 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="5db0f011-9979-4ecd-9525-5a02cfd6c8c8" containerName="istio-proxy" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.916226 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8a5b7b3-360d-475c-87b7-6a845394a09c" containerName="istio-proxy" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.916233 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="e3efd517-ed1d-4371-8009-856783d5a8c4" containerName="maas-api" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.916243 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8a5b7b3-360d-475c-87b7-6a845394a09c" containerName="istio-proxy" Nov 04 07:42:14.916814 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.916252 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="60ad1363-dd96-42e2-89ae-7c50adfae243" containerName="payload-pre-processing" Nov 04 07:42:14.921614 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.921581 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:14.932591 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:14.932562 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s"] Nov 04 07:42:15.024892 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.024842 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.024892 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.024898 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.025077 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.024993 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.025077 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.025016 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.025077 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.025042 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.025077 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.025063 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrtqm\" (UniqueName: \"kubernetes.io/projected/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-kube-api-access-lrtqm\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.025219 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.025169 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.025291 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.025269 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.025343 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.025316 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.126467 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.126433 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.126663 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.126475 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.126663 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.126494 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrtqm\" (UniqueName: \"kubernetes.io/projected/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-kube-api-access-lrtqm\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.126663 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.126550 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.126663 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.126616 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.126663 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.126660 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.126893 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.126829 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.126943 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.126887 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.126943 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.126930 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-workload-certs\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.127036 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.126998 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-workload-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.127036 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.127027 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.127280 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.127254 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-credential-socket\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.127424 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.127340 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-data\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.127424 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.127348 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istiod-ca-cert\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.129026 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.129002 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-envoy\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.129211 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.129195 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-podinfo\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.143542 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.143517 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrtqm\" (UniqueName: \"kubernetes.io/projected/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-kube-api-access-lrtqm\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.146198 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.146176 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-token\") pod \"e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.239424 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.239397 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:15.530164 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.530134 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s"] Nov 04 07:42:15.532456 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:42:15.532429 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedbafbc1_4f42_42d8_85b9_c3eb29ea6425.slice/crio-6ea2fd85f250cf8c7487c0b5e1329e94c7396b83283f7685101888ddbc4e899d WatchSource:0}: Error finding container 6ea2fd85f250cf8c7487c0b5e1329e94c7396b83283f7685101888ddbc4e899d: Status 404 returned error can't find the container with id 6ea2fd85f250cf8c7487c0b5e1329e94c7396b83283f7685101888ddbc4e899d Nov 04 07:42:15.534626 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.534596 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:42:15.534704 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.534675 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:42:15.534743 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.534704 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:42:15.864248 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.864158 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" event={"ID":"edbafbc1-4f42-42d8-85b9-c3eb29ea6425","Type":"ContainerStarted","Data":"0c95a5bb221eb374aabd3ba464ac686614d3af9c198dd3d59e8a5b3c95bc05b9"} Nov 04 07:42:15.864248 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.864224 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" event={"ID":"edbafbc1-4f42-42d8-85b9-c3eb29ea6425","Type":"ContainerStarted","Data":"6ea2fd85f250cf8c7487c0b5e1329e94c7396b83283f7685101888ddbc4e899d"} Nov 04 07:42:15.898402 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.895237 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" podStartSLOduration=1.895217937 podStartE2EDuration="1.895217937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:42:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:42:15.891641572 +0000 UTC m=+1571.225581209" watchObservedRunningTime="2026-11-04 07:42:15.895217937 +0000 UTC m=+1571.229157521" Nov 04 07:42:15.934269 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:15.932369 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:16.240450 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:16.240414 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:16.242117 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:16.242088 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Nov 04 07:42:16.242260 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:16.242139 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" podUID="edbafbc1-4f42-42d8-85b9-c3eb29ea6425" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Nov 04 07:42:16.609287 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:16.609201 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:16.736715 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:16.736635 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:16.873353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:16.873302 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:17.240058 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:17.239979 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Nov 04 07:42:17.240196 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:17.240055 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" podUID="edbafbc1-4f42-42d8-85b9-c3eb29ea6425" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Nov 04 07:42:17.785628 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:17.785595 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:18.240475 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:18.240437 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Nov 04 07:42:18.240861 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:18.240494 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" podUID="edbafbc1-4f42-42d8-85b9-c3eb29ea6425" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Nov 04 07:42:18.712047 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:18.712005 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:19.240378 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:19.240337 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Nov 04 07:42:19.240619 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:19.240396 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" podUID="edbafbc1-4f42-42d8-85b9-c3eb29ea6425" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Nov 04 07:42:19.689119 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:19.689080 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:19.913195 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:19.913158 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:20.062400 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:20.062299 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:20.241526 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:20.240886 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Nov 04 07:42:20.241526 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:20.240949 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" podUID="edbafbc1-4f42-42d8-85b9-c3eb29ea6425" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Nov 04 07:42:21.157467 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:21.157394 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:21.240473 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:21.240436 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Nov 04 07:42:21.240794 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:21.240755 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" podUID="edbafbc1-4f42-42d8-85b9-c3eb29ea6425" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Nov 04 07:42:21.289530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:21.284973 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:21.967479 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:21.967444 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p"] Nov 04 07:42:21.967789 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:21.967762 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" podUID="194ab3ed-32ab-4483-b60c-1c3c02451ee2" containerName="maas-api" containerID="cri-o://6dded2510d6a510ee9aa300cc311e76564a459476ecf6dda07f4d247a71a0a14" gracePeriod=30 Nov 04 07:42:22.010561 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.010476 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:22.262585 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.261392 5385 patch_prober.go:29] interesting pod/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" start-of-body= Nov 04 07:42:22.272524 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.263409 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:22.272524 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.263680 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" podUID="edbafbc1-4f42-42d8-85b9-c3eb29ea6425" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.80:15021/healthz/ready\": dial tcp 10.133.0.80:15021: connect: connection refused" Nov 04 07:42:22.338308 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.338281 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:42:22.354016 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.353985 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/194ab3ed-32ab-4483-b60c-1c3c02451ee2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/194ab3ed-32ab-4483-b60c-1c3c02451ee2-openshift-service-ca\") pod \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\" (UID: \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\") " Nov 04 07:42:22.354205 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.354055 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/194ab3ed-32ab-4483-b60c-1c3c02451ee2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/194ab3ed-32ab-4483-b60c-1c3c02451ee2-metrics-certs\") pod \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\" (UID: \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\") " Nov 04 07:42:22.354205 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.354161 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/194ab3ed-32ab-4483-b60c-1c3c02451ee2-kube-api-access-p7h79\" (UniqueName: \"kubernetes.io/projected/194ab3ed-32ab-4483-b60c-1c3c02451ee2-kube-api-access-p7h79\") pod \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\" (UID: \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\") " Nov 04 07:42:22.354334 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.354285 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/194ab3ed-32ab-4483-b60c-1c3c02451ee2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/194ab3ed-32ab-4483-b60c-1c3c02451ee2-maas-api-tls\") pod \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\" (UID: \"194ab3ed-32ab-4483-b60c-1c3c02451ee2\") " Nov 04 07:42:22.354447 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.354422 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/194ab3ed-32ab-4483-b60c-1c3c02451ee2-openshift-service-ca" pod "194ab3ed-32ab-4483-b60c-1c3c02451ee2" (UID: "194ab3ed-32ab-4483-b60c-1c3c02451ee2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:42:22.354756 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.354730 5385 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/194ab3ed-32ab-4483-b60c-1c3c02451ee2-openshift-service-ca\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:22.356395 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.356360 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/194ab3ed-32ab-4483-b60c-1c3c02451ee2-maas-api-tls" pod "194ab3ed-32ab-4483-b60c-1c3c02451ee2" (UID: "194ab3ed-32ab-4483-b60c-1c3c02451ee2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:42:22.356395 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.356375 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/194ab3ed-32ab-4483-b60c-1c3c02451ee2-metrics-certs" pod "194ab3ed-32ab-4483-b60c-1c3c02451ee2" (UID: "194ab3ed-32ab-4483-b60c-1c3c02451ee2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:42:22.356586 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.356448 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/194ab3ed-32ab-4483-b60c-1c3c02451ee2-kube-api-access-p7h79" pod "194ab3ed-32ab-4483-b60c-1c3c02451ee2" (UID: "194ab3ed-32ab-4483-b60c-1c3c02451ee2"). InnerVolumeSpecName "kube-api-access-p7h79". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:22.435247 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.435208 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:22.455799 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.455763 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7h79\" (UniqueName: \"kubernetes.io/projected/194ab3ed-32ab-4483-b60c-1c3c02451ee2-kube-api-access-p7h79\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:22.455799 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.455794 5385 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/194ab3ed-32ab-4483-b60c-1c3c02451ee2-maas-api-tls\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:22.455974 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.455808 5385 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/194ab3ed-32ab-4483-b60c-1c3c02451ee2-metrics-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:22.927248 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.927207 5385 generic.go:356] "Generic (PLEG): container finished" podID="194ab3ed-32ab-4483-b60c-1c3c02451ee2" containerID="6dded2510d6a510ee9aa300cc311e76564a459476ecf6dda07f4d247a71a0a14" exitCode=0 Nov 04 07:42:22.927451 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.927270 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" Nov 04 07:42:22.927451 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.927293 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" event={"ID":"194ab3ed-32ab-4483-b60c-1c3c02451ee2","Type":"ContainerDied","Data":"6dded2510d6a510ee9aa300cc311e76564a459476ecf6dda07f4d247a71a0a14"} Nov 04 07:42:22.927451 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.927361 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" event={"ID":"194ab3ed-32ab-4483-b60c-1c3c02451ee2","Type":"ContainerDied","Data":"03dc4490005f540925123bc8dd7127fa9be8390b1ff56b6779b4cce6d15528e5"} Nov 04 07:42:22.927451 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.927391 5385 scope.go:122] "RemoveContainer" containerID="6dded2510d6a510ee9aa300cc311e76564a459476ecf6dda07f4d247a71a0a14" Nov 04 07:42:22.937328 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.937301 5385 scope.go:122] "RemoveContainer" containerID="6dded2510d6a510ee9aa300cc311e76564a459476ecf6dda07f4d247a71a0a14" Nov 04 07:42:22.937639 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:22.937617 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6dded2510d6a510ee9aa300cc311e76564a459476ecf6dda07f4d247a71a0a14\": container with ID starting with 6dded2510d6a510ee9aa300cc311e76564a459476ecf6dda07f4d247a71a0a14 not found: ID does not exist" containerID="6dded2510d6a510ee9aa300cc311e76564a459476ecf6dda07f4d247a71a0a14" Nov 04 07:42:22.937717 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.937650 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dded2510d6a510ee9aa300cc311e76564a459476ecf6dda07f4d247a71a0a14"} err="failed to get container status \"6dded2510d6a510ee9aa300cc311e76564a459476ecf6dda07f4d247a71a0a14\": rpc error: code = NotFound desc = could not find container \"6dded2510d6a510ee9aa300cc311e76564a459476ecf6dda07f4d247a71a0a14\": container with ID starting with 6dded2510d6a510ee9aa300cc311e76564a459476ecf6dda07f4d247a71a0a14 not found: ID does not exist" Nov 04 07:42:22.947064 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.947001 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p"] Nov 04 07:42:22.949421 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:22.949398 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p"] Nov 04 07:42:23.356737 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:23.356695 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:23.668491 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:23.668450 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:23.669003 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:23.668981 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:23.670358 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:23.670336 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:23.792819 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:23.792787 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:23.915278 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:23.915243 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:24.034360 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:24.034236 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:24.201114 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:24.201075 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:24.478153 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:24.478116 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:24.812367 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:24.812289 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="194ab3ed-32ab-4483-b60c-1c3c02451ee2" path="/var/lib/kubelet/pods/194ab3ed-32ab-4483-b60c-1c3c02451ee2/volumes" Nov 04 07:42:25.259741 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:25.259702 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-a558a6-699b8cd5c9-47q6p" podUID="194ab3ed-32ab-4483-b60c-1c3c02451ee2" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.78:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 04 07:42:25.264046 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:25.264022 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:25.367535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:25.367489 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:25.864784 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:25.864749 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:26.134248 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.134212 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp"] Nov 04 07:42:26.134462 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.134436 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" podUID="5d21e3d9-a089-4eac-bbae-1be1ec3fc784" containerName="maas-api" containerID="cri-o://712c6cd3c87be1e18938360082595a80c3c62ef2cdc9a651ee3ab4b12abffe97" gracePeriod=30 Nov 04 07:42:26.203784 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.203754 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:26.209813 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.209783 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f"] Nov 04 07:42:26.210019 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.209990 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" podUID="f8cdda97-5400-4455-8741-9464bb3c430d" containerName="istio-proxy" containerID="cri-o://570d95fd00ecd2a7ef2aea092e24a2439b6e394443bd4838422941d1071afd27" gracePeriod=30 Nov 04 07:42:26.231896 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.231845 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw"] Nov 04 07:42:26.232374 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.232103 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" podUID="6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" containerName="istio-proxy" containerID="cri-o://e5b44ae6ca2862b0f5c19ada3b7f909caf3aa7fb55ddaee33ea51cfe3491e1e9" gracePeriod=30 Nov 04 07:42:26.236631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.236605 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8"] Nov 04 07:42:26.236812 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.236792 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" podUID="7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" containerName="istio-proxy" containerID="cri-o://7f974ff8777c22011a57274212e93046e281fac8dc300aeba771035aa6890817" gracePeriod=30 Nov 04 07:42:26.252133 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.252045 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s"] Nov 04 07:42:26.252253 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.252233 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" podUID="edbafbc1-4f42-42d8-85b9-c3eb29ea6425" containerName="istio-proxy" containerID="cri-o://0c95a5bb221eb374aabd3ba464ac686614d3af9c198dd3d59e8a5b3c95bc05b9" gracePeriod=30 Nov 04 07:42:26.325281 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.325246 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:26.473925 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.473840 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:42:26.551251 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.551209 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-metrics-certs\") pod \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\" (UID: \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\") " Nov 04 07:42:26.551445 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.551280 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-maas-api-tls\") pod \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\" (UID: \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\") " Nov 04 07:42:26.551445 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.551354 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-openshift-service-ca\") pod \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\" (UID: \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\") " Nov 04 07:42:26.551627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.551549 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-kube-api-access-fbjsh\" (UniqueName: \"kubernetes.io/projected/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-kube-api-access-fbjsh\") pod \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\" (UID: \"5d21e3d9-a089-4eac-bbae-1be1ec3fc784\") " Nov 04 07:42:26.551947 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.551913 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-openshift-service-ca" pod "5d21e3d9-a089-4eac-bbae-1be1ec3fc784" (UID: "5d21e3d9-a089-4eac-bbae-1be1ec3fc784"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:42:26.553845 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.553794 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-maas-api-tls" pod "5d21e3d9-a089-4eac-bbae-1be1ec3fc784" (UID: "5d21e3d9-a089-4eac-bbae-1be1ec3fc784"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:42:26.554211 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.554186 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-metrics-certs" pod "5d21e3d9-a089-4eac-bbae-1be1ec3fc784" (UID: "5d21e3d9-a089-4eac-bbae-1be1ec3fc784"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:42:26.554395 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.554376 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-kube-api-access-fbjsh" pod "5d21e3d9-a089-4eac-bbae-1be1ec3fc784" (UID: "5d21e3d9-a089-4eac-bbae-1be1ec3fc784"). InnerVolumeSpecName "kube-api-access-fbjsh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:26.653494 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.653452 5385 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-maas-api-tls\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:26.653494 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.653490 5385 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-openshift-service-ca\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:26.653494 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.653519 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbjsh\" (UniqueName: \"kubernetes.io/projected/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-kube-api-access-fbjsh\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:26.653767 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.653534 5385 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d21e3d9-a089-4eac-bbae-1be1ec3fc784-metrics-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:26.703567 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.703442 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:26.957145 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.957046 5385 generic.go:356] "Generic (PLEG): container finished" podID="5d21e3d9-a089-4eac-bbae-1be1ec3fc784" containerID="712c6cd3c87be1e18938360082595a80c3c62ef2cdc9a651ee3ab4b12abffe97" exitCode=0 Nov 04 07:42:26.957145 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.957083 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" Nov 04 07:42:26.957145 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.957116 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" event={"ID":"5d21e3d9-a089-4eac-bbae-1be1ec3fc784","Type":"ContainerDied","Data":"712c6cd3c87be1e18938360082595a80c3c62ef2cdc9a651ee3ab4b12abffe97"} Nov 04 07:42:26.957752 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.957166 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp" event={"ID":"5d21e3d9-a089-4eac-bbae-1be1ec3fc784","Type":"ContainerDied","Data":"2563aa3f73117be14f507bd82ccb6f4a0aaa97895e316ca2cc241c03b14a345a"} Nov 04 07:42:26.957752 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.957195 5385 scope.go:122] "RemoveContainer" containerID="712c6cd3c87be1e18938360082595a80c3c62ef2cdc9a651ee3ab4b12abffe97" Nov 04 07:42:26.966539 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.966492 5385 scope.go:122] "RemoveContainer" containerID="712c6cd3c87be1e18938360082595a80c3c62ef2cdc9a651ee3ab4b12abffe97" Nov 04 07:42:26.966848 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:26.966828 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"712c6cd3c87be1e18938360082595a80c3c62ef2cdc9a651ee3ab4b12abffe97\": container with ID starting with 712c6cd3c87be1e18938360082595a80c3c62ef2cdc9a651ee3ab4b12abffe97 not found: ID does not exist" containerID="712c6cd3c87be1e18938360082595a80c3c62ef2cdc9a651ee3ab4b12abffe97" Nov 04 07:42:26.966928 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.966854 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"712c6cd3c87be1e18938360082595a80c3c62ef2cdc9a651ee3ab4b12abffe97"} err="failed to get container status \"712c6cd3c87be1e18938360082595a80c3c62ef2cdc9a651ee3ab4b12abffe97\": rpc error: code = NotFound desc = could not find container \"712c6cd3c87be1e18938360082595a80c3c62ef2cdc9a651ee3ab4b12abffe97\": container with ID starting with 712c6cd3c87be1e18938360082595a80c3c62ef2cdc9a651ee3ab4b12abffe97 not found: ID does not exist" Nov 04 07:42:26.976530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.976492 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp"] Nov 04 07:42:26.979007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:26.978984 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-df3be8-d9988bdff-zncbp"] Nov 04 07:42:27.643783 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:27.643709 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:28.518401 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:28.518339 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:28.620196 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:28.620045 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:28.781577 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:28.781467 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:28.813639 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:28.813609 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5d21e3d9-a089-4eac-bbae-1be1ec3fc784" path="/var/lib/kubelet/pods/5d21e3d9-a089-4eac-bbae-1be1ec3fc784/volumes" Nov 04 07:42:29.463939 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:29.463899 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:29.582794 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:29.582702 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:29.690689 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:29.690644 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:29.972555 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:29.972498 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:30.321711 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:30.321633 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz"] Nov 04 07:42:30.321868 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:30.321844 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" podUID="63ab0edc-d9ba-486d-bfc0-7060678daea2" containerName="istio-proxy" containerID="cri-o://22dc93cc55bf4ca7de359d7177577e9d96ebcf20e0ee389496a80836719add1c" gracePeriod=30 Nov 04 07:42:30.360149 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:30.360119 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:30.690296 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:30.690258 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:30.866848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:30.866813 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:30.874933 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:30.874899 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:42:31.525954 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.525924 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:42:31.558579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.557612 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-podinfo\") pod \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " Nov 04 07:42:31.558579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.557671 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-token\" (UniqueName: \"kubernetes.io/projected/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-token\") pod \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " Nov 04 07:42:31.558579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.557721 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-workload-certs\") pod \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " Nov 04 07:42:31.558579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.557775 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-workload-socket\") pod \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " Nov 04 07:42:31.558579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.557835 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-kube-api-access-xt6w7\" (UniqueName: \"kubernetes.io/projected/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-kube-api-access-xt6w7\") pod \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " Nov 04 07:42:31.558579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.557871 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-credential-socket\") pod \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " Nov 04 07:42:31.558579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.557918 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istiod-ca-cert\") pod \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " Nov 04 07:42:31.558579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.557993 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-data\") pod \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " Nov 04 07:42:31.558579 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.558108 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-envoy\") pod \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\" (UID: \"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce\") " Nov 04 07:42:31.559290 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.559264 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-workload-certs" pod "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" (UID: "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.559652 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.559623 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-workload-socket" pod "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" (UID: "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.559781 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.559715 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-credential-socket" pod "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" (UID: "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.559781 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.559767 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-data" pod "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" (UID: "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.559961 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.559912 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istiod-ca-cert" pod "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" (UID: "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:42:31.562867 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.562842 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-podinfo" pod "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" (UID: "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 07:42:31.571135 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.569317 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-token" pod "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" (UID: "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:31.571135 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.569579 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-kube-api-access-xt6w7" pod "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" (UID: "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce"). InnerVolumeSpecName "kube-api-access-xt6w7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:31.577654 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.577612 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-envoy" pod "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" (UID: "7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.581368 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.581346 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:31.594922 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.594895 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:42:31.602873 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.602849 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:42:31.606025 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.606007 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:31.659555 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.659496 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-credential-socket\") pod \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " Nov 04 07:42:31.659734 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.659582 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-data\") pod \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " Nov 04 07:42:31.659734 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.659620 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-credential-socket\") pod \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " Nov 04 07:42:31.659734 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.659656 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f8cdda97-5400-4455-8741-9464bb3c430d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8cdda97-5400-4455-8741-9464bb3c430d-istiod-ca-cert\") pod \"f8cdda97-5400-4455-8741-9464bb3c430d\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " Nov 04 07:42:31.659734 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.659683 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istiod-ca-cert\") pod \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " Nov 04 07:42:31.659734 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.659715 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-istio-envoy\") pod \"f8cdda97-5400-4455-8741-9464bb3c430d\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " Nov 04 07:42:31.660222 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.659783 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-kube-api-access-lrtqm\" (UniqueName: \"kubernetes.io/projected/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-kube-api-access-lrtqm\") pod \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " Nov 04 07:42:31.660222 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.659831 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f8cdda97-5400-4455-8741-9464bb3c430d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8cdda97-5400-4455-8741-9464bb3c430d-istio-podinfo\") pod \"f8cdda97-5400-4455-8741-9464bb3c430d\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " Nov 04 07:42:31.660222 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.659877 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istiod-ca-cert\") pod \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " Nov 04 07:42:31.660222 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.659894 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-data" pod "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" (UID: "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.660222 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.659918 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-credential-socket" pod "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" (UID: "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.660222 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.659931 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-workload-certs\") pod \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " Nov 04 07:42:31.660222 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.659967 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-workload-certs\") pod \"f8cdda97-5400-4455-8741-9464bb3c430d\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " Nov 04 07:42:31.660222 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660008 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-token\" (UniqueName: \"kubernetes.io/projected/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-token\") pod \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " Nov 04 07:42:31.660222 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660045 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-workload-certs\") pod \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " Nov 04 07:42:31.660222 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660084 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-workload-socket\") pod \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " Nov 04 07:42:31.660222 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660124 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-data\") pod \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " Nov 04 07:42:31.660222 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660151 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-kube-api-access-lq46w\" (UniqueName: \"kubernetes.io/projected/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-kube-api-access-lq46w\") pod \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " Nov 04 07:42:31.660222 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660191 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-envoy\") pod \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " Nov 04 07:42:31.660962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660240 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-envoy\") pod \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " Nov 04 07:42:31.660962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660282 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-istio-data\") pod \"f8cdda97-5400-4455-8741-9464bb3c430d\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " Nov 04 07:42:31.660962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660321 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8cdda97-5400-4455-8741-9464bb3c430d-istiod-ca-cert" pod "f8cdda97-5400-4455-8741-9464bb3c430d" (UID: "f8cdda97-5400-4455-8741-9464bb3c430d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:42:31.660962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660338 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-podinfo\") pod \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " Nov 04 07:42:31.660962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660369 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-workload-socket\") pod \"f8cdda97-5400-4455-8741-9464bb3c430d\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " Nov 04 07:42:31.660962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660395 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8cdda97-5400-4455-8741-9464bb3c430d-istio-token\" (UniqueName: \"kubernetes.io/projected/f8cdda97-5400-4455-8741-9464bb3c430d-istio-token\") pod \"f8cdda97-5400-4455-8741-9464bb3c430d\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " Nov 04 07:42:31.660962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660429 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-token\" (UniqueName: \"kubernetes.io/projected/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-token\") pod \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " Nov 04 07:42:31.660962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660457 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-workload-socket\") pod \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\" (UID: \"edbafbc1-4f42-42d8-85b9-c3eb29ea6425\") " Nov 04 07:42:31.660962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660485 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-podinfo\") pod \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\" (UID: \"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a\") " Nov 04 07:42:31.660962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660554 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-credential-socket\") pod \"f8cdda97-5400-4455-8741-9464bb3c430d\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " Nov 04 07:42:31.660962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660581 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f8cdda97-5400-4455-8741-9464bb3c430d-kube-api-access-f6kdm\" (UniqueName: \"kubernetes.io/projected/f8cdda97-5400-4455-8741-9464bb3c430d-kube-api-access-f6kdm\") pod \"f8cdda97-5400-4455-8741-9464bb3c430d\" (UID: \"f8cdda97-5400-4455-8741-9464bb3c430d\") " Nov 04 07:42:31.660962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660600 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-credential-socket" pod "edbafbc1-4f42-42d8-85b9-c3eb29ea6425" (UID: "edbafbc1-4f42-42d8-85b9-c3eb29ea6425"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.660962 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.660909 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istiod-ca-cert" pod "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" (UID: "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:42:31.661631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661136 5385 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-data\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.661631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661157 5385 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-workload-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.661631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661173 5385 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-credential-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.661631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661188 5385 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f8cdda97-5400-4455-8741-9464bb3c430d-istiod-ca-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.661631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661188 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-workload-certs" pod "f8cdda97-5400-4455-8741-9464bb3c430d" (UID: "f8cdda97-5400-4455-8741-9464bb3c430d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.661631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661204 5385 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istiod-ca-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.661631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661219 5385 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-workload-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.661631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661234 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xt6w7\" (UniqueName: \"kubernetes.io/projected/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-kube-api-access-xt6w7\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.661631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661249 5385 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-credential-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.661631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661265 5385 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istiod-ca-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.661631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661281 5385 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-data\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.661631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661295 5385 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-envoy\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.661631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661309 5385 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-podinfo\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.661631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661323 5385 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce-istio-token\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.661631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661337 5385 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-credential-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.661631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661546 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-workload-certs" pod "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" (UID: "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.661631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661571 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-workload-socket" pod "f8cdda97-5400-4455-8741-9464bb3c430d" (UID: "f8cdda97-5400-4455-8741-9464bb3c430d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.662409 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.661778 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-workload-socket" pod "edbafbc1-4f42-42d8-85b9-c3eb29ea6425" (UID: "edbafbc1-4f42-42d8-85b9-c3eb29ea6425"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.664303 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.664122 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-credential-socket" pod "f8cdda97-5400-4455-8741-9464bb3c430d" (UID: "f8cdda97-5400-4455-8741-9464bb3c430d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.664428 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.664405 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istiod-ca-cert" pod "edbafbc1-4f42-42d8-85b9-c3eb29ea6425" (UID: "edbafbc1-4f42-42d8-85b9-c3eb29ea6425"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:42:31.665331 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.664650 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-workload-certs" pod "edbafbc1-4f42-42d8-85b9-c3eb29ea6425" (UID: "edbafbc1-4f42-42d8-85b9-c3eb29ea6425"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.665331 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.665210 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-data" pod "edbafbc1-4f42-42d8-85b9-c3eb29ea6425" (UID: "edbafbc1-4f42-42d8-85b9-c3eb29ea6425"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.665331 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.665285 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-workload-socket" pod "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" (UID: "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.665995 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.665974 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-istio-data" pod "f8cdda97-5400-4455-8741-9464bb3c430d" (UID: "f8cdda97-5400-4455-8741-9464bb3c430d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.666327 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.666308 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-envoy" pod "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" (UID: "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.666431 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.666305 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8cdda97-5400-4455-8741-9464bb3c430d-istio-token" pod "f8cdda97-5400-4455-8741-9464bb3c430d" (UID: "f8cdda97-5400-4455-8741-9464bb3c430d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:31.666431 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.666326 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-token" pod "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" (UID: "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:31.666785 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.666760 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8cdda97-5400-4455-8741-9464bb3c430d-kube-api-access-f6kdm" pod "f8cdda97-5400-4455-8741-9464bb3c430d" (UID: "f8cdda97-5400-4455-8741-9464bb3c430d"). InnerVolumeSpecName "kube-api-access-f6kdm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:31.666893 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.666786 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-podinfo" pod "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" (UID: "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 07:42:31.667908 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.667886 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-kube-api-access-lq46w" pod "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" (UID: "6a2a80ac-2bde-4025-9a4f-db2daa2ba28a"). InnerVolumeSpecName "kube-api-access-lq46w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:31.668674 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.668286 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-kube-api-access-lrtqm" pod "edbafbc1-4f42-42d8-85b9-c3eb29ea6425" (UID: "edbafbc1-4f42-42d8-85b9-c3eb29ea6425"). InnerVolumeSpecName "kube-api-access-lrtqm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:31.668674 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.668359 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-envoy" pod "edbafbc1-4f42-42d8-85b9-c3eb29ea6425" (UID: "edbafbc1-4f42-42d8-85b9-c3eb29ea6425"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.668674 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.668425 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-podinfo" pod "edbafbc1-4f42-42d8-85b9-c3eb29ea6425" (UID: "edbafbc1-4f42-42d8-85b9-c3eb29ea6425"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 07:42:31.668674 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.668497 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f8cdda97-5400-4455-8741-9464bb3c430d-istio-podinfo" pod "f8cdda97-5400-4455-8741-9464bb3c430d" (UID: "f8cdda97-5400-4455-8741-9464bb3c430d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 07:42:31.668674 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.668645 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-token" pod "edbafbc1-4f42-42d8-85b9-c3eb29ea6425" (UID: "edbafbc1-4f42-42d8-85b9-c3eb29ea6425"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:31.669303 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.669278 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-istio-envoy" pod "f8cdda97-5400-4455-8741-9464bb3c430d" (UID: "f8cdda97-5400-4455-8741-9464bb3c430d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:31.762126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762041 5385 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-workload-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762077 5385 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-workload-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762088 5385 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-token\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762097 5385 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-workload-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762107 5385 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-workload-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762116 5385 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-data\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762126 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762125 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lq46w\" (UniqueName: \"kubernetes.io/projected/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-kube-api-access-lq46w\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762134 5385 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-envoy\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762142 5385 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-envoy\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762150 5385 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-istio-data\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762158 5385 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istio-podinfo\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762166 5385 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-workload-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762176 5385 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f8cdda97-5400-4455-8741-9464bb3c430d-istio-token\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762184 5385 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-token\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762193 5385 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-workload-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762201 5385 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a-istio-podinfo\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762211 5385 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-credential-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762220 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6kdm\" (UniqueName: \"kubernetes.io/projected/f8cdda97-5400-4455-8741-9464bb3c430d-kube-api-access-f6kdm\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762228 5385 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f8cdda97-5400-4455-8741-9464bb3c430d-istio-envoy\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762236 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrtqm\" (UniqueName: \"kubernetes.io/projected/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-kube-api-access-lrtqm\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762244 5385 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f8cdda97-5400-4455-8741-9464bb3c430d-istio-podinfo\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.762691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.762252 5385 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/edbafbc1-4f42-42d8-85b9-c3eb29ea6425-istiod-ca-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:31.986133 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.986095 5385 generic.go:356] "Generic (PLEG): container finished" podID="f8cdda97-5400-4455-8741-9464bb3c430d" containerID="570d95fd00ecd2a7ef2aea092e24a2439b6e394443bd4838422941d1071afd27" exitCode=0 Nov 04 07:42:31.986286 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.986141 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" Nov 04 07:42:31.986286 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.986184 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" event={"ID":"f8cdda97-5400-4455-8741-9464bb3c430d","Type":"ContainerDied","Data":"570d95fd00ecd2a7ef2aea092e24a2439b6e394443bd4838422941d1071afd27"} Nov 04 07:42:31.986286 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.986242 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f" event={"ID":"f8cdda97-5400-4455-8741-9464bb3c430d","Type":"ContainerDied","Data":"0be047a5a99d3930f5f7147cafcc83849a15308ca3c6db8614ed72e132cbee55"} Nov 04 07:42:31.986286 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.986263 5385 scope.go:122] "RemoveContainer" containerID="570d95fd00ecd2a7ef2aea092e24a2439b6e394443bd4838422941d1071afd27" Nov 04 07:42:31.987774 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.987752 5385 generic.go:356] "Generic (PLEG): container finished" podID="edbafbc1-4f42-42d8-85b9-c3eb29ea6425" containerID="0c95a5bb221eb374aabd3ba464ac686614d3af9c198dd3d59e8a5b3c95bc05b9" exitCode=0 Nov 04 07:42:31.987869 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.987800 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" Nov 04 07:42:31.987869 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.987841 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" event={"ID":"edbafbc1-4f42-42d8-85b9-c3eb29ea6425","Type":"ContainerDied","Data":"0c95a5bb221eb374aabd3ba464ac686614d3af9c198dd3d59e8a5b3c95bc05b9"} Nov 04 07:42:31.987987 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.987884 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s" event={"ID":"edbafbc1-4f42-42d8-85b9-c3eb29ea6425","Type":"ContainerDied","Data":"6ea2fd85f250cf8c7487c0b5e1329e94c7396b83283f7685101888ddbc4e899d"} Nov 04 07:42:31.989747 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.989724 5385 generic.go:356] "Generic (PLEG): container finished" podID="6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" containerID="e5b44ae6ca2862b0f5c19ada3b7f909caf3aa7fb55ddaee33ea51cfe3491e1e9" exitCode=0 Nov 04 07:42:31.989861 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.989783 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" Nov 04 07:42:31.989861 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.989812 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" event={"ID":"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a","Type":"ContainerDied","Data":"e5b44ae6ca2862b0f5c19ada3b7f909caf3aa7fb55ddaee33ea51cfe3491e1e9"} Nov 04 07:42:31.989971 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.989866 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw" event={"ID":"6a2a80ac-2bde-4025-9a4f-db2daa2ba28a","Type":"ContainerDied","Data":"fb15afeabd4718f0c4e4e4fc429448ef0fabbcb1d831f0f404fc54d4406d0559"} Nov 04 07:42:31.991611 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.991587 5385 generic.go:356] "Generic (PLEG): container finished" podID="7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" containerID="7f974ff8777c22011a57274212e93046e281fac8dc300aeba771035aa6890817" exitCode=0 Nov 04 07:42:31.991726 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.991634 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" Nov 04 07:42:31.991726 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.991664 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" event={"ID":"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce","Type":"ContainerDied","Data":"7f974ff8777c22011a57274212e93046e281fac8dc300aeba771035aa6890817"} Nov 04 07:42:31.991726 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:31.991702 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8" event={"ID":"7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce","Type":"ContainerDied","Data":"43163122b7b39a53a976cec71982e93a41ba84176364c345b88b632b4490d7fb"} Nov 04 07:42:32.003458 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.003432 5385 scope.go:122] "RemoveContainer" containerID="570d95fd00ecd2a7ef2aea092e24a2439b6e394443bd4838422941d1071afd27" Nov 04 07:42:32.003809 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:32.003785 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"570d95fd00ecd2a7ef2aea092e24a2439b6e394443bd4838422941d1071afd27\": container with ID starting with 570d95fd00ecd2a7ef2aea092e24a2439b6e394443bd4838422941d1071afd27 not found: ID does not exist" containerID="570d95fd00ecd2a7ef2aea092e24a2439b6e394443bd4838422941d1071afd27" Nov 04 07:42:32.003894 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.003822 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"570d95fd00ecd2a7ef2aea092e24a2439b6e394443bd4838422941d1071afd27"} err="failed to get container status \"570d95fd00ecd2a7ef2aea092e24a2439b6e394443bd4838422941d1071afd27\": rpc error: code = NotFound desc = could not find container \"570d95fd00ecd2a7ef2aea092e24a2439b6e394443bd4838422941d1071afd27\": container with ID starting with 570d95fd00ecd2a7ef2aea092e24a2439b6e394443bd4838422941d1071afd27 not found: ID does not exist" Nov 04 07:42:32.003894 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.003848 5385 scope.go:122] "RemoveContainer" containerID="0c95a5bb221eb374aabd3ba464ac686614d3af9c198dd3d59e8a5b3c95bc05b9" Nov 04 07:42:32.013464 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.013448 5385 scope.go:122] "RemoveContainer" containerID="0c95a5bb221eb374aabd3ba464ac686614d3af9c198dd3d59e8a5b3c95bc05b9" Nov 04 07:42:32.013754 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:32.013734 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c95a5bb221eb374aabd3ba464ac686614d3af9c198dd3d59e8a5b3c95bc05b9\": container with ID starting with 0c95a5bb221eb374aabd3ba464ac686614d3af9c198dd3d59e8a5b3c95bc05b9 not found: ID does not exist" containerID="0c95a5bb221eb374aabd3ba464ac686614d3af9c198dd3d59e8a5b3c95bc05b9" Nov 04 07:42:32.013830 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.013760 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c95a5bb221eb374aabd3ba464ac686614d3af9c198dd3d59e8a5b3c95bc05b9"} err="failed to get container status \"0c95a5bb221eb374aabd3ba464ac686614d3af9c198dd3d59e8a5b3c95bc05b9\": rpc error: code = NotFound desc = could not find container \"0c95a5bb221eb374aabd3ba464ac686614d3af9c198dd3d59e8a5b3c95bc05b9\": container with ID starting with 0c95a5bb221eb374aabd3ba464ac686614d3af9c198dd3d59e8a5b3c95bc05b9 not found: ID does not exist" Nov 04 07:42:32.013830 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.013776 5385 scope.go:122] "RemoveContainer" containerID="e5b44ae6ca2862b0f5c19ada3b7f909caf3aa7fb55ddaee33ea51cfe3491e1e9" Nov 04 07:42:32.022371 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.022355 5385 scope.go:122] "RemoveContainer" containerID="b1688dc4967fb416014424c6cf890f716885edfa81f525a40b96a3c97cf9cfc8" Nov 04 07:42:32.022798 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.022777 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f"] Nov 04 07:42:32.031438 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.031421 5385 scope.go:122] "RemoveContainer" containerID="e5b44ae6ca2862b0f5c19ada3b7f909caf3aa7fb55ddaee33ea51cfe3491e1e9" Nov 04 07:42:32.031823 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.031799 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-2kb2f"] Nov 04 07:42:32.031823 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:32.031812 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5b44ae6ca2862b0f5c19ada3b7f909caf3aa7fb55ddaee33ea51cfe3491e1e9\": container with ID starting with e5b44ae6ca2862b0f5c19ada3b7f909caf3aa7fb55ddaee33ea51cfe3491e1e9 not found: ID does not exist" containerID="e5b44ae6ca2862b0f5c19ada3b7f909caf3aa7fb55ddaee33ea51cfe3491e1e9" Nov 04 07:42:32.031964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.031841 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5b44ae6ca2862b0f5c19ada3b7f909caf3aa7fb55ddaee33ea51cfe3491e1e9"} err="failed to get container status \"e5b44ae6ca2862b0f5c19ada3b7f909caf3aa7fb55ddaee33ea51cfe3491e1e9\": rpc error: code = NotFound desc = could not find container \"e5b44ae6ca2862b0f5c19ada3b7f909caf3aa7fb55ddaee33ea51cfe3491e1e9\": container with ID starting with e5b44ae6ca2862b0f5c19ada3b7f909caf3aa7fb55ddaee33ea51cfe3491e1e9 not found: ID does not exist" Nov 04 07:42:32.031964 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.031858 5385 scope.go:122] "RemoveContainer" containerID="b1688dc4967fb416014424c6cf890f716885edfa81f525a40b96a3c97cf9cfc8" Nov 04 07:42:32.032130 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:32.032111 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1688dc4967fb416014424c6cf890f716885edfa81f525a40b96a3c97cf9cfc8\": container with ID starting with b1688dc4967fb416014424c6cf890f716885edfa81f525a40b96a3c97cf9cfc8 not found: ID does not exist" containerID="b1688dc4967fb416014424c6cf890f716885edfa81f525a40b96a3c97cf9cfc8" Nov 04 07:42:32.032211 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.032138 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1688dc4967fb416014424c6cf890f716885edfa81f525a40b96a3c97cf9cfc8"} err="failed to get container status \"b1688dc4967fb416014424c6cf890f716885edfa81f525a40b96a3c97cf9cfc8\": rpc error: code = NotFound desc = could not find container \"b1688dc4967fb416014424c6cf890f716885edfa81f525a40b96a3c97cf9cfc8\": container with ID starting with b1688dc4967fb416014424c6cf890f716885edfa81f525a40b96a3c97cf9cfc8 not found: ID does not exist" Nov 04 07:42:32.032211 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.032154 5385 scope.go:122] "RemoveContainer" containerID="7f974ff8777c22011a57274212e93046e281fac8dc300aeba771035aa6890817" Nov 04 07:42:32.040338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.040318 5385 scope.go:122] "RemoveContainer" containerID="7f974ff8777c22011a57274212e93046e281fac8dc300aeba771035aa6890817" Nov 04 07:42:32.040633 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:32.040615 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f974ff8777c22011a57274212e93046e281fac8dc300aeba771035aa6890817\": container with ID starting with 7f974ff8777c22011a57274212e93046e281fac8dc300aeba771035aa6890817 not found: ID does not exist" containerID="7f974ff8777c22011a57274212e93046e281fac8dc300aeba771035aa6890817" Nov 04 07:42:32.040694 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.040639 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f974ff8777c22011a57274212e93046e281fac8dc300aeba771035aa6890817"} err="failed to get container status \"7f974ff8777c22011a57274212e93046e281fac8dc300aeba771035aa6890817\": rpc error: code = NotFound desc = could not find container \"7f974ff8777c22011a57274212e93046e281fac8dc300aeba771035aa6890817\": container with ID starting with 7f974ff8777c22011a57274212e93046e281fac8dc300aeba771035aa6890817 not found: ID does not exist" Nov 04 07:42:32.054787 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.054758 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw"] Nov 04 07:42:32.069604 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.069563 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-db2sw"] Nov 04 07:42:32.070493 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.070471 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx"] Nov 04 07:42:32.070936 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.070922 5385 cpu_manager.go:414] "containerMap: removing container" podUID="7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" containerName="istio-proxy" Nov 04 07:42:32.070992 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.070937 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" containerName="istio-proxy" Nov 04 07:42:32.070992 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.070949 5385 cpu_manager.go:414] "containerMap: removing container" podUID="5d21e3d9-a089-4eac-bbae-1be1ec3fc784" containerName="maas-api" Nov 04 07:42:32.070992 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.070954 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5d21e3d9-a089-4eac-bbae-1be1ec3fc784" containerName="maas-api" Nov 04 07:42:32.070992 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.070968 5385 cpu_manager.go:414] "containerMap: removing container" podUID="f8cdda97-5400-4455-8741-9464bb3c430d" containerName="istio-proxy" Nov 04 07:42:32.070992 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.070977 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f8cdda97-5400-4455-8741-9464bb3c430d" containerName="istio-proxy" Nov 04 07:42:32.070992 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.070993 5385 cpu_manager.go:414] "containerMap: removing container" podUID="6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" containerName="istio-proxy" Nov 04 07:42:32.071242 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.071000 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" containerName="istio-proxy" Nov 04 07:42:32.071242 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.071014 5385 cpu_manager.go:414] "containerMap: removing container" podUID="194ab3ed-32ab-4483-b60c-1c3c02451ee2" containerName="maas-api" Nov 04 07:42:32.071242 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.071020 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="194ab3ed-32ab-4483-b60c-1c3c02451ee2" containerName="maas-api" Nov 04 07:42:32.071242 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.071058 5385 cpu_manager.go:414] "containerMap: removing container" podUID="edbafbc1-4f42-42d8-85b9-c3eb29ea6425" containerName="istio-proxy" Nov 04 07:42:32.071242 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.071064 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="edbafbc1-4f42-42d8-85b9-c3eb29ea6425" containerName="istio-proxy" Nov 04 07:42:32.071242 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.071180 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" containerName="istio-proxy" Nov 04 07:42:32.071242 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.071194 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="edbafbc1-4f42-42d8-85b9-c3eb29ea6425" containerName="istio-proxy" Nov 04 07:42:32.071242 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.071204 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" containerName="istio-proxy" Nov 04 07:42:32.071242 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.071213 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="5d21e3d9-a089-4eac-bbae-1be1ec3fc784" containerName="maas-api" Nov 04 07:42:32.071242 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.071222 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="f8cdda97-5400-4455-8741-9464bb3c430d" containerName="istio-proxy" Nov 04 07:42:32.071242 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.071232 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="194ab3ed-32ab-4483-b60c-1c3c02451ee2" containerName="maas-api" Nov 04 07:42:32.076291 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.076270 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.082699 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.082674 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z"] Nov 04 07:42:32.083306 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.083285 5385 cpu_manager.go:414] "containerMap: removing container" podUID="6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" containerName="istio-proxy" Nov 04 07:42:32.083411 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.083308 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" containerName="istio-proxy" Nov 04 07:42:32.083480 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.083448 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" containerName="istio-proxy" Nov 04 07:42:32.086630 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.086609 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.118851 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.114050 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx"] Nov 04 07:42:32.118851 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.118580 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8"] Nov 04 07:42:32.121287 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.121216 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z"] Nov 04 07:42:32.124402 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.124318 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-27dg8"] Nov 04 07:42:32.126058 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.126005 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s"] Nov 04 07:42:32.127610 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.127589 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-016e6c-openshift-default-644bc689bc-hfl9s"] Nov 04 07:42:32.165606 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.165575 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97sdc\" (UniqueName: \"kubernetes.io/projected/a0faffc2-896c-4420-b382-47ecdcb49abb-kube-api-access-97sdc\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.165745 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.165623 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.165745 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.165707 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.165745 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.165725 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.165864 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.165747 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfw4h\" (UniqueName: \"kubernetes.io/projected/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-kube-api-access-qfw4h\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.165864 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.165849 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.165931 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.165874 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.165931 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.165918 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.165990 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.165946 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.165990 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.165975 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.166056 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.166005 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.166056 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.166046 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.166115 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.166062 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.166115 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.166082 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.166185 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.166163 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.166223 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.166204 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.166254 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.166224 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0faffc2-896c-4420-b382-47ecdcb49abb-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.166296 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.166285 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.266940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.266869 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0faffc2-896c-4420-b382-47ecdcb49abb-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.266940 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.266911 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.267139 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.266946 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97sdc\" (UniqueName: \"kubernetes.io/projected/a0faffc2-896c-4420-b382-47ecdcb49abb-kube-api-access-97sdc\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.267139 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.266978 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.267139 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267005 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.267139 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267028 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.267345 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267196 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfw4h\" (UniqueName: \"kubernetes.io/projected/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-kube-api-access-qfw4h\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.267345 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267250 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.267448 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267408 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.267525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267451 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.267525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267451 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-workload-certs\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.267525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267477 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.267687 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267531 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.267687 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267572 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.267687 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267596 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.267687 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267621 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.267687 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267665 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.267945 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267688 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.267945 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267720 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.267945 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267725 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0faffc2-896c-4420-b382-47ecdcb49abb-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.267945 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267727 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.267945 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267769 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.267945 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267802 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.267945 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267851 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.268253 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.267953 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-workload-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.268253 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.268058 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-data\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.268253 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.268059 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-credential-socket\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.268543 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.268520 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istiod-ca-cert\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.270226 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.270206 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.270386 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.270370 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-envoy\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.270535 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.270492 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.270575 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.270496 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-podinfo\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.279944 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.279897 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfw4h\" (UniqueName: \"kubernetes.io/projected/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-kube-api-access-qfw4h\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.279944 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.279913 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.280272 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.280206 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-token\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.280339 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.280290 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97sdc\" (UniqueName: \"kubernetes.io/projected/a0faffc2-896c-4420-b382-47ecdcb49abb-kube-api-access-97sdc\") pod \"e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.414567 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.414524 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:32.422976 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.422955 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:32.583458 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.583372 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:32.707483 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.707450 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx"] Nov 04 07:42:32.709659 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:42:32.709634 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc52ab653_d09f_4e9b_bfa3_b37acc4ea0d8.slice/crio-3b30898e3142314ad279296e8512649626425be53c100e6730e93a34f7cd0aa3 WatchSource:0}: Error finding container 3b30898e3142314ad279296e8512649626425be53c100e6730e93a34f7cd0aa3: Status 404 returned error can't find the container with id 3b30898e3142314ad279296e8512649626425be53c100e6730e93a34f7cd0aa3 Nov 04 07:42:32.711443 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.711421 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z"] Nov 04 07:42:32.712319 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.712285 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:42:32.712422 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.712364 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:42:32.712422 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.712409 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:42:32.712692 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:42:32.712669 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0faffc2_896c_4420_b382_47ecdcb49abb.slice/crio-323f81a30c1a8f0a53dcebd3f59e7aafc711cb746270227faeb0dac062c24781 WatchSource:0}: Error finding container 323f81a30c1a8f0a53dcebd3f59e7aafc711cb746270227faeb0dac062c24781: Status 404 returned error can't find the container with id 323f81a30c1a8f0a53dcebd3f59e7aafc711cb746270227faeb0dac062c24781 Nov 04 07:42:32.714863 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.714837 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:42:32.714929 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.714911 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:42:32.715015 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.715000 5385 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Nov 04 07:42:32.820206 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.820175 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6a2a80ac-2bde-4025-9a4f-db2daa2ba28a" path="/var/lib/kubelet/pods/6a2a80ac-2bde-4025-9a4f-db2daa2ba28a/volumes" Nov 04 07:42:32.820952 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.820824 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce" path="/var/lib/kubelet/pods/7e2400ce-b1f1-48a9-b0bb-be239c8dd0ce/volumes" Nov 04 07:42:32.821380 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.821362 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="edbafbc1-4f42-42d8-85b9-c3eb29ea6425" path="/var/lib/kubelet/pods/edbafbc1-4f42-42d8-85b9-c3eb29ea6425/volumes" Nov 04 07:42:32.822175 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.822156 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f8cdda97-5400-4455-8741-9464bb3c430d" path="/var/lib/kubelet/pods/f8cdda97-5400-4455-8741-9464bb3c430d/volumes" Nov 04 07:42:32.822917 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.822889 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:32.927619 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:32.927584 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:33.004250 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:33.004210 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" event={"ID":"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8","Type":"ContainerStarted","Data":"d4c5b04d7f972bab0eb1e68eba91304b59dd3e0d6e81ee2481dad145b4267060"} Nov 04 07:42:33.004421 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:33.004273 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" event={"ID":"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8","Type":"ContainerStarted","Data":"3b30898e3142314ad279296e8512649626425be53c100e6730e93a34f7cd0aa3"} Nov 04 07:42:33.006127 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:33.006094 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" event={"ID":"a0faffc2-896c-4420-b382-47ecdcb49abb","Type":"ContainerStarted","Data":"fdc457cb7de65c18adf5ce38c84f1b6778e42494c73a6c925bd8b2184ad3f666"} Nov 04 07:42:33.006267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:33.006137 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" event={"ID":"a0faffc2-896c-4420-b382-47ecdcb49abb","Type":"ContainerStarted","Data":"323f81a30c1a8f0a53dcebd3f59e7aafc711cb746270227faeb0dac062c24781"} Nov 04 07:42:33.030138 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:33.030073 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" podStartSLOduration=1.030054371 podStartE2EDuration="1.030054371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:42:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:42:33.027369803 +0000 UTC m=+1588.361309411" watchObservedRunningTime="2026-11-04 07:42:33.030054371 +0000 UTC m=+1588.363993957" Nov 04 07:42:33.052464 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:33.052407 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" podStartSLOduration=1.052388795 podStartE2EDuration="1.052388795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:42:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:42:33.049966612 +0000 UTC m=+1588.383906197" watchObservedRunningTime="2026-11-04 07:42:33.052388795 +0000 UTC m=+1588.386328374" Nov 04 07:42:33.415326 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:33.415288 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:33.417605 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:33.417576 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body= Nov 04 07:42:33.417735 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:33.417640 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" podUID="c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" Nov 04 07:42:33.423973 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:33.423950 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:33.425089 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:33.425069 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body= Nov 04 07:42:33.425181 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:33.425125 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" podUID="a0faffc2-896c-4420-b382-47ecdcb49abb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" Nov 04 07:42:33.480685 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:33.480655 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:34.415226 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:34.415196 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body= Nov 04 07:42:34.415732 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:34.415706 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" podUID="c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" Nov 04 07:42:34.423819 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:34.423776 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body= Nov 04 07:42:34.423928 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:34.423838 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" podUID="a0faffc2-896c-4420-b382-47ecdcb49abb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" Nov 04 07:42:34.526848 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:34.526814 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:35.415262 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.415225 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body= Nov 04 07:42:35.415681 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.415289 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" podUID="c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" Nov 04 07:42:35.423700 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.423678 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body= Nov 04 07:42:35.423754 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.423726 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" podUID="a0faffc2-896c-4420-b382-47ecdcb49abb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" Nov 04 07:42:35.721899 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.721874 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:42:35.745199 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.745173 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-data\") pod \"63ab0edc-d9ba-486d-bfc0-7060678daea2\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " Nov 04 07:42:35.745332 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.745288 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63ab0edc-d9ba-486d-bfc0-7060678daea2-kube-api-access-d75sg\" (UniqueName: \"kubernetes.io/projected/63ab0edc-d9ba-486d-bfc0-7060678daea2-kube-api-access-d75sg\") pod \"63ab0edc-d9ba-486d-bfc0-7060678daea2\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " Nov 04 07:42:35.745374 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.745328 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-credential-socket\") pod \"63ab0edc-d9ba-486d-bfc0-7060678daea2\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " Nov 04 07:42:35.745374 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.745358 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-podinfo\") pod \"63ab0edc-d9ba-486d-bfc0-7060678daea2\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " Nov 04 07:42:35.745436 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.745374 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-envoy\") pod \"63ab0edc-d9ba-486d-bfc0-7060678daea2\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " Nov 04 07:42:35.745436 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.745396 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63ab0edc-d9ba-486d-bfc0-7060678daea2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63ab0edc-d9ba-486d-bfc0-7060678daea2-istiod-ca-cert\") pod \"63ab0edc-d9ba-486d-bfc0-7060678daea2\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " Nov 04 07:42:35.745436 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.745417 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-workload-socket\") pod \"63ab0edc-d9ba-486d-bfc0-7060678daea2\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " Nov 04 07:42:35.745436 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.745417 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-data" pod "63ab0edc-d9ba-486d-bfc0-7060678daea2" (UID: "63ab0edc-d9ba-486d-bfc0-7060678daea2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:35.745436 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.745435 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-token\" (UniqueName: \"kubernetes.io/projected/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-token\") pod \"63ab0edc-d9ba-486d-bfc0-7060678daea2\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " Nov 04 07:42:35.745676 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.745479 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-workload-certs\") pod \"63ab0edc-d9ba-486d-bfc0-7060678daea2\" (UID: \"63ab0edc-d9ba-486d-bfc0-7060678daea2\") " Nov 04 07:42:35.759552 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.745854 5385 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-data\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:35.759713 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.759586 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-workload-certs" pod "63ab0edc-d9ba-486d-bfc0-7060678daea2" (UID: "63ab0edc-d9ba-486d-bfc0-7060678daea2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:35.801769 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.801403 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:35.802838 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.802805 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-credential-socket" pod "63ab0edc-d9ba-486d-bfc0-7060678daea2" (UID: "63ab0edc-d9ba-486d-bfc0-7060678daea2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:35.803027 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.803004 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-workload-socket" pod "63ab0edc-d9ba-486d-bfc0-7060678daea2" (UID: "63ab0edc-d9ba-486d-bfc0-7060678daea2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:35.803325 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.803303 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63ab0edc-d9ba-486d-bfc0-7060678daea2-istiod-ca-cert" pod "63ab0edc-d9ba-486d-bfc0-7060678daea2" (UID: "63ab0edc-d9ba-486d-bfc0-7060678daea2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:42:35.817685 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.806671 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-podinfo" pod "63ab0edc-d9ba-486d-bfc0-7060678daea2" (UID: "63ab0edc-d9ba-486d-bfc0-7060678daea2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 07:42:35.817685 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.807282 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-token" pod "63ab0edc-d9ba-486d-bfc0-7060678daea2" (UID: "63ab0edc-d9ba-486d-bfc0-7060678daea2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:35.817685 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.807388 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63ab0edc-d9ba-486d-bfc0-7060678daea2-kube-api-access-d75sg" pod "63ab0edc-d9ba-486d-bfc0-7060678daea2" (UID: "63ab0edc-d9ba-486d-bfc0-7060678daea2"). InnerVolumeSpecName "kube-api-access-d75sg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:35.817685 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.808083 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-envoy" pod "63ab0edc-d9ba-486d-bfc0-7060678daea2" (UID: "63ab0edc-d9ba-486d-bfc0-7060678daea2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:35.849528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.849221 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d75sg\" (UniqueName: \"kubernetes.io/projected/63ab0edc-d9ba-486d-bfc0-7060678daea2-kube-api-access-d75sg\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:35.849528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.849270 5385 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-credential-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:35.849528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.849289 5385 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-podinfo\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:35.849528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.849303 5385 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-envoy\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:35.849528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.849317 5385 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63ab0edc-d9ba-486d-bfc0-7060678daea2-istiod-ca-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:35.849528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.849332 5385 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-workload-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:35.849528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.849346 5385 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63ab0edc-d9ba-486d-bfc0-7060678daea2-istio-token\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:35.849528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.849361 5385 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63ab0edc-d9ba-486d-bfc0-7060678daea2-workload-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:35.966434 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:35.966214 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:36.042832 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.042799 5385 generic.go:356] "Generic (PLEG): container finished" podID="63ab0edc-d9ba-486d-bfc0-7060678daea2" containerID="22dc93cc55bf4ca7de359d7177577e9d96ebcf20e0ee389496a80836719add1c" exitCode=0 Nov 04 07:42:36.043002 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.042882 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" event={"ID":"63ab0edc-d9ba-486d-bfc0-7060678daea2","Type":"ContainerDied","Data":"22dc93cc55bf4ca7de359d7177577e9d96ebcf20e0ee389496a80836719add1c"} Nov 04 07:42:36.043002 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.042921 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" event={"ID":"63ab0edc-d9ba-486d-bfc0-7060678daea2","Type":"ContainerDied","Data":"de2071b7c1906f2a02f052588024bcf3ffaba16d19508b3f8f71a0d065409dfb"} Nov 04 07:42:36.043002 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.042943 5385 scope.go:122] "RemoveContainer" containerID="22dc93cc55bf4ca7de359d7177577e9d96ebcf20e0ee389496a80836719add1c" Nov 04 07:42:36.043113 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.043049 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz" Nov 04 07:42:36.117178 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.117153 5385 scope.go:122] "RemoveContainer" containerID="19a36bedda2349d3371d0983319133ed846dd32d1e66982ff1cb0c7af13512f2" Nov 04 07:42:36.125685 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.125666 5385 scope.go:122] "RemoveContainer" containerID="22dc93cc55bf4ca7de359d7177577e9d96ebcf20e0ee389496a80836719add1c" Nov 04 07:42:36.125966 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:36.125943 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22dc93cc55bf4ca7de359d7177577e9d96ebcf20e0ee389496a80836719add1c\": container with ID starting with 22dc93cc55bf4ca7de359d7177577e9d96ebcf20e0ee389496a80836719add1c not found: ID does not exist" containerID="22dc93cc55bf4ca7de359d7177577e9d96ebcf20e0ee389496a80836719add1c" Nov 04 07:42:36.126043 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.125980 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22dc93cc55bf4ca7de359d7177577e9d96ebcf20e0ee389496a80836719add1c"} err="failed to get container status \"22dc93cc55bf4ca7de359d7177577e9d96ebcf20e0ee389496a80836719add1c\": rpc error: code = NotFound desc = could not find container \"22dc93cc55bf4ca7de359d7177577e9d96ebcf20e0ee389496a80836719add1c\": container with ID starting with 22dc93cc55bf4ca7de359d7177577e9d96ebcf20e0ee389496a80836719add1c not found: ID does not exist" Nov 04 07:42:36.126043 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.126008 5385 scope.go:122] "RemoveContainer" containerID="19a36bedda2349d3371d0983319133ed846dd32d1e66982ff1cb0c7af13512f2" Nov 04 07:42:36.126290 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:36.126270 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19a36bedda2349d3371d0983319133ed846dd32d1e66982ff1cb0c7af13512f2\": container with ID starting with 19a36bedda2349d3371d0983319133ed846dd32d1e66982ff1cb0c7af13512f2 not found: ID does not exist" containerID="19a36bedda2349d3371d0983319133ed846dd32d1e66982ff1cb0c7af13512f2" Nov 04 07:42:36.126330 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.126296 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19a36bedda2349d3371d0983319133ed846dd32d1e66982ff1cb0c7af13512f2"} err="failed to get container status \"19a36bedda2349d3371d0983319133ed846dd32d1e66982ff1cb0c7af13512f2\": rpc error: code = NotFound desc = could not find container \"19a36bedda2349d3371d0983319133ed846dd32d1e66982ff1cb0c7af13512f2\": container with ID starting with 19a36bedda2349d3371d0983319133ed846dd32d1e66982ff1cb0c7af13512f2 not found: ID does not exist" Nov 04 07:42:36.154538 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.148027 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz"] Nov 04 07:42:36.188531 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.187788 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-a558a6-openshift-default-57d8959bdf-4tzfz"] Nov 04 07:42:36.415210 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.415171 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body= Nov 04 07:42:36.415366 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.415226 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" podUID="c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" Nov 04 07:42:36.423922 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.423899 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body= Nov 04 07:42:36.424007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.423943 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" podUID="a0faffc2-896c-4420-b382-47ecdcb49abb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" Nov 04 07:42:36.443609 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.443581 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:36.812644 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.812612 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63ab0edc-d9ba-486d-bfc0-7060678daea2" path="/var/lib/kubelet/pods/63ab0edc-d9ba-486d-bfc0-7060678daea2/volumes" Nov 04 07:42:36.866476 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:36.866439 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:37.415764 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:37.415729 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body= Nov 04 07:42:37.416148 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:37.415784 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" podUID="c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" Nov 04 07:42:37.423253 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:37.423227 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body= Nov 04 07:42:37.423381 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:37.423271 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" podUID="a0faffc2-896c-4420-b382-47ecdcb49abb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" Nov 04 07:42:37.937047 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:37.937009 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:38.416788 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:38.416754 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body= Nov 04 07:42:38.417138 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:38.416819 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" podUID="c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" Nov 04 07:42:38.423603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:38.423563 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body= Nov 04 07:42:38.423745 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:38.423620 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" podUID="a0faffc2-896c-4420-b382-47ecdcb49abb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" Nov 04 07:42:38.932009 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:38.931972 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:39.414959 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:39.414907 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" start-of-body= Nov 04 07:42:39.415141 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:39.414980 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" podUID="c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.81:15021/healthz/ready\": dial tcp 10.133.0.81:15021: connect: connection refused" Nov 04 07:42:39.423306 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:39.423278 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" start-of-body= Nov 04 07:42:39.423731 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:39.423325 5385 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" podUID="a0faffc2-896c-4420-b382-47ecdcb49abb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.82:15021/healthz/ready\": dial tcp 10.133.0.82:15021: connect: connection refused" Nov 04 07:42:39.980264 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:39.980202 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:40.087528 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:40.086984 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:40.901702 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:40.901647 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:40.994480 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:40.990632 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:40.994480 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:40.990974 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:41.074729 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:41.074702 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:41.119928 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:41.119895 5385 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:41.120827 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:41.120780 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:41.121694 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:41.121671 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:41.783359 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:41.783304 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:42.512098 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:42.512046 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:42.613119 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:42.613085 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:43.180092 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:43.180062 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:43.279780 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:43.279733 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:44.117073 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:44.117032 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:44.327525 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:44.324124 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:44.449532 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:44.449468 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:44.581786 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:44.581748 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq"] Nov 04 07:42:44.582029 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:44.582002 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" podUID="2b293d12-35ef-47df-9c77-27f734441df4" containerName="istio-proxy" containerID="cri-o://72ec7d8e1bf19d4f7f53cbbe4bbe2d7d77b3d1680c4fbfbf80382ad629b48eaa" gracePeriod=30 Nov 04 07:42:44.590497 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:44.590025 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc"] Nov 04 07:42:44.590497 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:44.590360 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" podUID="99d41a69-904b-44b9-9a38-253d859dac34" containerName="istio-proxy" containerID="cri-o://51e74e1e865eebec1c3095c6162bccb6191ffa5b84db749f4d481f4d0fbe1f9c" gracePeriod=30 Nov 04 07:42:44.598700 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:44.598673 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx"] Nov 04 07:42:44.598875 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:44.598855 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" podUID="c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" containerName="istio-proxy" containerID="cri-o://d4c5b04d7f972bab0eb1e68eba91304b59dd3e0d6e81ee2481dad145b4267060" gracePeriod=30 Nov 04 07:42:44.609691 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:44.609669 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z"] Nov 04 07:42:44.609851 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:44.609831 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" podUID="a0faffc2-896c-4420-b382-47ecdcb49abb" containerName="istio-proxy" containerID="cri-o://fdc457cb7de65c18adf5ce38c84f1b6778e42494c73a6c925bd8b2184ad3f666" gracePeriod=30 Nov 04 07:42:44.638133 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:44.638105 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:47.638333 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:47.638293 5385 patch_prober.go:29] interesting pod/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 07:42:47.638708 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:47.638352 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" podUID="2b293d12-35ef-47df-9c77-27f734441df4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 07:42:49.867105 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.867085 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:42:49.948830 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.948804 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:42:49.952806 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.952785 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:49.957697 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.957674 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:49.981467 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.979993 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2b293d12-35ef-47df-9c77-27f734441df4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b293d12-35ef-47df-9c77-27f734441df4-istiod-ca-cert\") pod \"2b293d12-35ef-47df-9c77-27f734441df4\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " Nov 04 07:42:49.981467 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.980471 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-istio-envoy\") pod \"2b293d12-35ef-47df-9c77-27f734441df4\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " Nov 04 07:42:49.981467 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.980543 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-workload-socket\") pod \"2b293d12-35ef-47df-9c77-27f734441df4\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " Nov 04 07:42:49.981467 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.980618 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b293d12-35ef-47df-9c77-27f734441df4-istio-token\" (UniqueName: \"kubernetes.io/projected/2b293d12-35ef-47df-9c77-27f734441df4-istio-token\") pod \"2b293d12-35ef-47df-9c77-27f734441df4\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " Nov 04 07:42:49.981467 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.980651 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-istio-data\") pod \"2b293d12-35ef-47df-9c77-27f734441df4\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " Nov 04 07:42:49.981467 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.980699 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2b293d12-35ef-47df-9c77-27f734441df4-kube-api-access-pf5gg\" (UniqueName: \"kubernetes.io/projected/2b293d12-35ef-47df-9c77-27f734441df4-kube-api-access-pf5gg\") pod \"2b293d12-35ef-47df-9c77-27f734441df4\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " Nov 04 07:42:49.981467 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.980801 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-credential-socket\") pod \"2b293d12-35ef-47df-9c77-27f734441df4\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " Nov 04 07:42:49.981467 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.980854 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-workload-certs\") pod \"2b293d12-35ef-47df-9c77-27f734441df4\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " Nov 04 07:42:49.981467 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.980894 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2b293d12-35ef-47df-9c77-27f734441df4-istiod-ca-cert" pod "2b293d12-35ef-47df-9c77-27f734441df4" (UID: "2b293d12-35ef-47df-9c77-27f734441df4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:42:49.981467 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.980911 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2b293d12-35ef-47df-9c77-27f734441df4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b293d12-35ef-47df-9c77-27f734441df4-istio-podinfo\") pod \"2b293d12-35ef-47df-9c77-27f734441df4\" (UID: \"2b293d12-35ef-47df-9c77-27f734441df4\") " Nov 04 07:42:49.981467 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.981145 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-istio-data" pod "2b293d12-35ef-47df-9c77-27f734441df4" (UID: "2b293d12-35ef-47df-9c77-27f734441df4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:49.981467 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.981270 5385 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2b293d12-35ef-47df-9c77-27f734441df4-istiod-ca-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:49.981467 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.981281 5385 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-istio-data\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:49.982598 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.982532 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-credential-socket" pod "2b293d12-35ef-47df-9c77-27f734441df4" (UID: "2b293d12-35ef-47df-9c77-27f734441df4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:49.983381 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.982855 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-workload-certs" pod "2b293d12-35ef-47df-9c77-27f734441df4" (UID: "2b293d12-35ef-47df-9c77-27f734441df4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:49.983381 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.983039 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-workload-socket" pod "2b293d12-35ef-47df-9c77-27f734441df4" (UID: "2b293d12-35ef-47df-9c77-27f734441df4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:49.983610 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.983588 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2b293d12-35ef-47df-9c77-27f734441df4-istio-podinfo" pod "2b293d12-35ef-47df-9c77-27f734441df4" (UID: "2b293d12-35ef-47df-9c77-27f734441df4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 07:42:49.984398 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.984376 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-istio-envoy" pod "2b293d12-35ef-47df-9c77-27f734441df4" (UID: "2b293d12-35ef-47df-9c77-27f734441df4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:49.984457 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.984432 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b293d12-35ef-47df-9c77-27f734441df4-kube-api-access-pf5gg" pod "2b293d12-35ef-47df-9c77-27f734441df4" (UID: "2b293d12-35ef-47df-9c77-27f734441df4"). InnerVolumeSpecName "kube-api-access-pf5gg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:49.984802 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:49.984788 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b293d12-35ef-47df-9c77-27f734441df4-istio-token" pod "2b293d12-35ef-47df-9c77-27f734441df4" (UID: "2b293d12-35ef-47df-9c77-27f734441df4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:50.034951 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.034892 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:50.081667 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.081637 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0faffc2-896c-4420-b382-47ecdcb49abb-kube-api-access-97sdc\" (UniqueName: \"kubernetes.io/projected/a0faffc2-896c-4420-b382-47ecdcb49abb-kube-api-access-97sdc\") pod \"a0faffc2-896c-4420-b382-47ecdcb49abb\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " Nov 04 07:42:50.081667 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.081669 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-podinfo\") pod \"a0faffc2-896c-4420-b382-47ecdcb49abb\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " Nov 04 07:42:50.081907 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.081689 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-credential-socket\") pod \"99d41a69-904b-44b9-9a38-253d859dac34\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " Nov 04 07:42:50.081907 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.081721 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istiod-ca-cert\") pod \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " Nov 04 07:42:50.081907 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.081757 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/99d41a69-904b-44b9-9a38-253d859dac34-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99d41a69-904b-44b9-9a38-253d859dac34-istiod-ca-cert\") pod \"99d41a69-904b-44b9-9a38-253d859dac34\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " Nov 04 07:42:50.081907 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.081780 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-envoy\") pod \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " Nov 04 07:42:50.081907 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.081831 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99d41a69-904b-44b9-9a38-253d859dac34-istio-token\" (UniqueName: \"kubernetes.io/projected/99d41a69-904b-44b9-9a38-253d859dac34-istio-token\") pod \"99d41a69-904b-44b9-9a38-253d859dac34\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " Nov 04 07:42:50.081907 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.081865 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-workload-certs\") pod \"99d41a69-904b-44b9-9a38-253d859dac34\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " Nov 04 07:42:50.081907 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.081897 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-workload-certs\") pod \"a0faffc2-896c-4420-b382-47ecdcb49abb\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " Nov 04 07:42:50.082267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.081935 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-credential-socket\") pod \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " Nov 04 07:42:50.082267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.081971 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-envoy\") pod \"a0faffc2-896c-4420-b382-47ecdcb49abb\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " Nov 04 07:42:50.082267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.081981 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-credential-socket" pod "99d41a69-904b-44b9-9a38-253d859dac34" (UID: "99d41a69-904b-44b9-9a38-253d859dac34"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:50.082267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082007 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-workload-socket\") pod \"a0faffc2-896c-4420-b382-47ecdcb49abb\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " Nov 04 07:42:50.082267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082041 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-istio-envoy\") pod \"99d41a69-904b-44b9-9a38-253d859dac34\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " Nov 04 07:42:50.082267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082088 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-data\") pod \"a0faffc2-896c-4420-b382-47ecdcb49abb\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " Nov 04 07:42:50.082267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082104 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istiod-ca-cert" pod "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" (UID: "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:42:50.082267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082116 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-podinfo\") pod \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " Nov 04 07:42:50.082267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082173 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-workload-socket\") pod \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " Nov 04 07:42:50.082267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082210 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-token\" (UniqueName: \"kubernetes.io/projected/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-token\") pod \"a0faffc2-896c-4420-b382-47ecdcb49abb\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " Nov 04 07:42:50.082267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082222 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-workload-certs" pod "99d41a69-904b-44b9-9a38-253d859dac34" (UID: "99d41a69-904b-44b9-9a38-253d859dac34"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:50.082267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082240 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-workload-certs\") pod \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " Nov 04 07:42:50.082267 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082269 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-token\" (UniqueName: \"kubernetes.io/projected/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-token\") pod \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " Nov 04 07:42:50.082948 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082297 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a0faffc2-896c-4420-b382-47ecdcb49abb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0faffc2-896c-4420-b382-47ecdcb49abb-istiod-ca-cert\") pod \"a0faffc2-896c-4420-b382-47ecdcb49abb\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " Nov 04 07:42:50.082948 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082331 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-credential-socket\") pod \"a0faffc2-896c-4420-b382-47ecdcb49abb\" (UID: \"a0faffc2-896c-4420-b382-47ecdcb49abb\") " Nov 04 07:42:50.082948 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082367 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-kube-api-access-qfw4h\" (UniqueName: \"kubernetes.io/projected/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-kube-api-access-qfw4h\") pod \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " Nov 04 07:42:50.082948 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082410 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/99d41a69-904b-44b9-9a38-253d859dac34-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99d41a69-904b-44b9-9a38-253d859dac34-istio-podinfo\") pod \"99d41a69-904b-44b9-9a38-253d859dac34\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " Nov 04 07:42:50.082948 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082445 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-istio-data\") pod \"99d41a69-904b-44b9-9a38-253d859dac34\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " Nov 04 07:42:50.082948 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082528 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/99d41a69-904b-44b9-9a38-253d859dac34-kube-api-access-d5dr5\" (UniqueName: \"kubernetes.io/projected/99d41a69-904b-44b9-9a38-253d859dac34-kube-api-access-d5dr5\") pod \"99d41a69-904b-44b9-9a38-253d859dac34\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " Nov 04 07:42:50.082948 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082556 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-workload-socket\") pod \"99d41a69-904b-44b9-9a38-253d859dac34\" (UID: \"99d41a69-904b-44b9-9a38-253d859dac34\") " Nov 04 07:42:50.082948 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082594 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-data\") pod \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\" (UID: \"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8\") " Nov 04 07:42:50.083364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.083044 5385 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-credential-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.083364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.083067 5385 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-workload-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.083364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.083084 5385 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-workload-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.083364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.083100 5385 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2b293d12-35ef-47df-9c77-27f734441df4-istio-podinfo\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.083364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.083114 5385 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-istio-envoy\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.083364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.083135 5385 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2b293d12-35ef-47df-9c77-27f734441df4-workload-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.083364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.083148 5385 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2b293d12-35ef-47df-9c77-27f734441df4-istio-token\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.083364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.083161 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf5gg\" (UniqueName: \"kubernetes.io/projected/2b293d12-35ef-47df-9c77-27f734441df4-kube-api-access-pf5gg\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.083364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.083175 5385 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-credential-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.083364 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.083189 5385 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istiod-ca-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.083870 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082344 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-credential-socket" pod "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" (UID: "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:50.083870 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082376 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-workload-certs" pod "a0faffc2-896c-4420-b382-47ecdcb49abb" (UID: "a0faffc2-896c-4420-b382-47ecdcb49abb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:50.083870 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.082559 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99d41a69-904b-44b9-9a38-253d859dac34-istiod-ca-cert" pod "99d41a69-904b-44b9-9a38-253d859dac34" (UID: "99d41a69-904b-44b9-9a38-253d859dac34"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:42:50.083870 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.083207 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-workload-socket" pod "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" (UID: "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:50.083870 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.083393 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-data" pod "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" (UID: "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:50.083870 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.083692 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-workload-certs" pod "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" (UID: "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:50.084155 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.083928 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-workload-socket" pod "a0faffc2-896c-4420-b382-47ecdcb49abb" (UID: "a0faffc2-896c-4420-b382-47ecdcb49abb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:50.084155 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.083979 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-data" pod "a0faffc2-896c-4420-b382-47ecdcb49abb" (UID: "a0faffc2-896c-4420-b382-47ecdcb49abb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:50.084542 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.084478 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-credential-socket" pod "a0faffc2-896c-4420-b382-47ecdcb49abb" (UID: "a0faffc2-896c-4420-b382-47ecdcb49abb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:50.084542 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.084535 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-workload-socket" pod "99d41a69-904b-44b9-9a38-253d859dac34" (UID: "99d41a69-904b-44b9-9a38-253d859dac34"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:50.084728 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.084644 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-istio-data" pod "99d41a69-904b-44b9-9a38-253d859dac34" (UID: "99d41a69-904b-44b9-9a38-253d859dac34"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:50.084787 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.084677 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0faffc2-896c-4420-b382-47ecdcb49abb-istiod-ca-cert" pod "a0faffc2-896c-4420-b382-47ecdcb49abb" (UID: "a0faffc2-896c-4420-b382-47ecdcb49abb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:42:50.085024 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.084996 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-podinfo" pod "a0faffc2-896c-4420-b382-47ecdcb49abb" (UID: "a0faffc2-896c-4420-b382-47ecdcb49abb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 07:42:50.086019 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.085993 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/99d41a69-904b-44b9-9a38-253d859dac34-istio-podinfo" pod "99d41a69-904b-44b9-9a38-253d859dac34" (UID: "99d41a69-904b-44b9-9a38-253d859dac34"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 07:42:50.086172 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.086130 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0faffc2-896c-4420-b382-47ecdcb49abb-kube-api-access-97sdc" pod "a0faffc2-896c-4420-b382-47ecdcb49abb" (UID: "a0faffc2-896c-4420-b382-47ecdcb49abb"). InnerVolumeSpecName "kube-api-access-97sdc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:50.086401 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.086360 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-envoy" pod "a0faffc2-896c-4420-b382-47ecdcb49abb" (UID: "a0faffc2-896c-4420-b382-47ecdcb49abb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:50.086926 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.086901 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-podinfo" pod "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" (UID: "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 07:42:50.087008 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.086929 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-istio-envoy" pod "99d41a69-904b-44b9-9a38-253d859dac34" (UID: "99d41a69-904b-44b9-9a38-253d859dac34"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:50.087777 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.087749 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-envoy" pod "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" (UID: "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:42:50.087878 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.087774 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99d41a69-904b-44b9-9a38-253d859dac34-istio-token" pod "99d41a69-904b-44b9-9a38-253d859dac34" (UID: "99d41a69-904b-44b9-9a38-253d859dac34"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:50.087878 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.087808 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-token" pod "a0faffc2-896c-4420-b382-47ecdcb49abb" (UID: "a0faffc2-896c-4420-b382-47ecdcb49abb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:50.087878 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.087820 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-token" pod "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" (UID: "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:50.088639 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.088616 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99d41a69-904b-44b9-9a38-253d859dac34-kube-api-access-d5dr5" pod "99d41a69-904b-44b9-9a38-253d859dac34" (UID: "99d41a69-904b-44b9-9a38-253d859dac34"). InnerVolumeSpecName "kube-api-access-d5dr5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:50.089028 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.089009 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-kube-api-access-qfw4h" pod "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" (UID: "c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8"). InnerVolumeSpecName "kube-api-access-qfw4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:50.134286 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.134261 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:50.183401 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183373 5385 generic.go:356] "Generic (PLEG): container finished" podID="99d41a69-904b-44b9-9a38-253d859dac34" containerID="51e74e1e865eebec1c3095c6162bccb6191ffa5b84db749f4d481f4d0fbe1f9c" exitCode=0 Nov 04 07:42:50.183550 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183425 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" Nov 04 07:42:50.183550 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183472 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" event={"ID":"99d41a69-904b-44b9-9a38-253d859dac34","Type":"ContainerDied","Data":"51e74e1e865eebec1c3095c6162bccb6191ffa5b84db749f4d481f4d0fbe1f9c"} Nov 04 07:42:50.183660 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183551 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc" event={"ID":"99d41a69-904b-44b9-9a38-253d859dac34","Type":"ContainerDied","Data":"63d8101c232663f8de1289334efa2ad2c9e68d23f8522b0f3f45e616cb5309b4"} Nov 04 07:42:50.183660 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183583 5385 scope.go:122] "RemoveContainer" containerID="51e74e1e865eebec1c3095c6162bccb6191ffa5b84db749f4d481f4d0fbe1f9c" Nov 04 07:42:50.183827 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183806 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97sdc\" (UniqueName: \"kubernetes.io/projected/a0faffc2-896c-4420-b382-47ecdcb49abb-kube-api-access-97sdc\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.183899 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183831 5385 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-podinfo\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.183899 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183847 5385 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/99d41a69-904b-44b9-9a38-253d859dac34-istiod-ca-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.183899 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183859 5385 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-envoy\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.183899 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183868 5385 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99d41a69-904b-44b9-9a38-253d859dac34-istio-token\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.183899 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183876 5385 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-workload-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.183899 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183887 5385 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-credential-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.184097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183898 5385 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-envoy\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.184097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183917 5385 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-workload-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.184097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183930 5385 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-istio-envoy\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.184097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183945 5385 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-data\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.184097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183954 5385 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-podinfo\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.184097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183965 5385 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-workload-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.184097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183977 5385 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0faffc2-896c-4420-b382-47ecdcb49abb-istio-token\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.184097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183989 5385 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-workload-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.184097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.183998 5385 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-token\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.184097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.184006 5385 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0faffc2-896c-4420-b382-47ecdcb49abb-istiod-ca-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.184097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.184015 5385 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0faffc2-896c-4420-b382-47ecdcb49abb-credential-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.184097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.184029 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfw4h\" (UniqueName: \"kubernetes.io/projected/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-kube-api-access-qfw4h\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.184097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.184041 5385 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/99d41a69-904b-44b9-9a38-253d859dac34-istio-podinfo\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.184097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.184049 5385 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-istio-data\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.184097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.184057 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5dr5\" (UniqueName: \"kubernetes.io/projected/99d41a69-904b-44b9-9a38-253d859dac34-kube-api-access-d5dr5\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.184097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.184066 5385 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/99d41a69-904b-44b9-9a38-253d859dac34-workload-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.184097 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.184074 5385 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8-istio-data\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:50.194266 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.194240 5385 generic.go:356] "Generic (PLEG): container finished" podID="2b293d12-35ef-47df-9c77-27f734441df4" containerID="72ec7d8e1bf19d4f7f53cbbe4bbe2d7d77b3d1680c4fbfbf80382ad629b48eaa" exitCode=0 Nov 04 07:42:50.194369 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.194328 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" Nov 04 07:42:50.194856 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.194388 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" event={"ID":"2b293d12-35ef-47df-9c77-27f734441df4","Type":"ContainerDied","Data":"72ec7d8e1bf19d4f7f53cbbe4bbe2d7d77b3d1680c4fbfbf80382ad629b48eaa"} Nov 04 07:42:50.194856 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.194470 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq" event={"ID":"2b293d12-35ef-47df-9c77-27f734441df4","Type":"ContainerDied","Data":"b188858963392e946285fe7b0b1160e14cf23e96f0e8d80cff362a5b4819e93a"} Nov 04 07:42:50.196035 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.196015 5385 generic.go:356] "Generic (PLEG): container finished" podID="c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" containerID="d4c5b04d7f972bab0eb1e68eba91304b59dd3e0d6e81ee2481dad145b4267060" exitCode=0 Nov 04 07:42:50.196135 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.196044 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" event={"ID":"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8","Type":"ContainerDied","Data":"d4c5b04d7f972bab0eb1e68eba91304b59dd3e0d6e81ee2481dad145b4267060"} Nov 04 07:42:50.196135 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.196061 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" Nov 04 07:42:50.196135 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.196097 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx" event={"ID":"c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8","Type":"ContainerDied","Data":"3b30898e3142314ad279296e8512649626425be53c100e6730e93a34f7cd0aa3"} Nov 04 07:42:50.197801 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.197780 5385 generic.go:356] "Generic (PLEG): container finished" podID="a0faffc2-896c-4420-b382-47ecdcb49abb" containerID="fdc457cb7de65c18adf5ce38c84f1b6778e42494c73a6c925bd8b2184ad3f666" exitCode=0 Nov 04 07:42:50.197886 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.197812 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" event={"ID":"a0faffc2-896c-4420-b382-47ecdcb49abb","Type":"ContainerDied","Data":"fdc457cb7de65c18adf5ce38c84f1b6778e42494c73a6c925bd8b2184ad3f666"} Nov 04 07:42:50.197886 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.197840 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" Nov 04 07:42:50.198005 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.197846 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z" event={"ID":"a0faffc2-896c-4420-b382-47ecdcb49abb","Type":"ContainerDied","Data":"323f81a30c1a8f0a53dcebd3f59e7aafc711cb746270227faeb0dac062c24781"} Nov 04 07:42:50.202269 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.202246 5385 scope.go:122] "RemoveContainer" containerID="51e74e1e865eebec1c3095c6162bccb6191ffa5b84db749f4d481f4d0fbe1f9c" Nov 04 07:42:50.202609 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:50.202588 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51e74e1e865eebec1c3095c6162bccb6191ffa5b84db749f4d481f4d0fbe1f9c\": container with ID starting with 51e74e1e865eebec1c3095c6162bccb6191ffa5b84db749f4d481f4d0fbe1f9c not found: ID does not exist" containerID="51e74e1e865eebec1c3095c6162bccb6191ffa5b84db749f4d481f4d0fbe1f9c" Nov 04 07:42:50.202671 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.202621 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51e74e1e865eebec1c3095c6162bccb6191ffa5b84db749f4d481f4d0fbe1f9c"} err="failed to get container status \"51e74e1e865eebec1c3095c6162bccb6191ffa5b84db749f4d481f4d0fbe1f9c\": rpc error: code = NotFound desc = could not find container \"51e74e1e865eebec1c3095c6162bccb6191ffa5b84db749f4d481f4d0fbe1f9c\": container with ID starting with 51e74e1e865eebec1c3095c6162bccb6191ffa5b84db749f4d481f4d0fbe1f9c not found: ID does not exist" Nov 04 07:42:50.202671 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.202642 5385 scope.go:122] "RemoveContainer" containerID="72ec7d8e1bf19d4f7f53cbbe4bbe2d7d77b3d1680c4fbfbf80382ad629b48eaa" Nov 04 07:42:50.212201 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.212185 5385 scope.go:122] "RemoveContainer" containerID="ee8270bfe828eda79d75ab27e52c190d1294cab114932de683a68ac2dcf9ac8b" Nov 04 07:42:50.219640 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.219620 5385 scope.go:122] "RemoveContainer" containerID="72ec7d8e1bf19d4f7f53cbbe4bbe2d7d77b3d1680c4fbfbf80382ad629b48eaa" Nov 04 07:42:50.219870 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:50.219853 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72ec7d8e1bf19d4f7f53cbbe4bbe2d7d77b3d1680c4fbfbf80382ad629b48eaa\": container with ID starting with 72ec7d8e1bf19d4f7f53cbbe4bbe2d7d77b3d1680c4fbfbf80382ad629b48eaa not found: ID does not exist" containerID="72ec7d8e1bf19d4f7f53cbbe4bbe2d7d77b3d1680c4fbfbf80382ad629b48eaa" Nov 04 07:42:50.219967 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.219876 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72ec7d8e1bf19d4f7f53cbbe4bbe2d7d77b3d1680c4fbfbf80382ad629b48eaa"} err="failed to get container status \"72ec7d8e1bf19d4f7f53cbbe4bbe2d7d77b3d1680c4fbfbf80382ad629b48eaa\": rpc error: code = NotFound desc = could not find container \"72ec7d8e1bf19d4f7f53cbbe4bbe2d7d77b3d1680c4fbfbf80382ad629b48eaa\": container with ID starting with 72ec7d8e1bf19d4f7f53cbbe4bbe2d7d77b3d1680c4fbfbf80382ad629b48eaa not found: ID does not exist" Nov 04 07:42:50.219967 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.219891 5385 scope.go:122] "RemoveContainer" containerID="ee8270bfe828eda79d75ab27e52c190d1294cab114932de683a68ac2dcf9ac8b" Nov 04 07:42:50.220147 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:50.220132 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee8270bfe828eda79d75ab27e52c190d1294cab114932de683a68ac2dcf9ac8b\": container with ID starting with ee8270bfe828eda79d75ab27e52c190d1294cab114932de683a68ac2dcf9ac8b not found: ID does not exist" containerID="ee8270bfe828eda79d75ab27e52c190d1294cab114932de683a68ac2dcf9ac8b" Nov 04 07:42:50.220241 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.220151 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee8270bfe828eda79d75ab27e52c190d1294cab114932de683a68ac2dcf9ac8b"} err="failed to get container status \"ee8270bfe828eda79d75ab27e52c190d1294cab114932de683a68ac2dcf9ac8b\": rpc error: code = NotFound desc = could not find container \"ee8270bfe828eda79d75ab27e52c190d1294cab114932de683a68ac2dcf9ac8b\": container with ID starting with ee8270bfe828eda79d75ab27e52c190d1294cab114932de683a68ac2dcf9ac8b not found: ID does not exist" Nov 04 07:42:50.220241 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.220166 5385 scope.go:122] "RemoveContainer" containerID="d4c5b04d7f972bab0eb1e68eba91304b59dd3e0d6e81ee2481dad145b4267060" Nov 04 07:42:50.223016 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.222999 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq"] Nov 04 07:42:50.229554 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.229533 5385 scope.go:122] "RemoveContainer" containerID="d4c5b04d7f972bab0eb1e68eba91304b59dd3e0d6e81ee2481dad145b4267060" Nov 04 07:42:50.229648 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.229571 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-kvtcq"] Nov 04 07:42:50.229832 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:50.229812 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4c5b04d7f972bab0eb1e68eba91304b59dd3e0d6e81ee2481dad145b4267060\": container with ID starting with d4c5b04d7f972bab0eb1e68eba91304b59dd3e0d6e81ee2481dad145b4267060 not found: ID does not exist" containerID="d4c5b04d7f972bab0eb1e68eba91304b59dd3e0d6e81ee2481dad145b4267060" Nov 04 07:42:50.229894 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.229836 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4c5b04d7f972bab0eb1e68eba91304b59dd3e0d6e81ee2481dad145b4267060"} err="failed to get container status \"d4c5b04d7f972bab0eb1e68eba91304b59dd3e0d6e81ee2481dad145b4267060\": rpc error: code = NotFound desc = could not find container \"d4c5b04d7f972bab0eb1e68eba91304b59dd3e0d6e81ee2481dad145b4267060\": container with ID starting with d4c5b04d7f972bab0eb1e68eba91304b59dd3e0d6e81ee2481dad145b4267060 not found: ID does not exist" Nov 04 07:42:50.229894 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.229851 5385 scope.go:122] "RemoveContainer" containerID="fdc457cb7de65c18adf5ce38c84f1b6778e42494c73a6c925bd8b2184ad3f666" Nov 04 07:42:50.237649 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.237633 5385 scope.go:122] "RemoveContainer" containerID="fdc457cb7de65c18adf5ce38c84f1b6778e42494c73a6c925bd8b2184ad3f666" Nov 04 07:42:50.237932 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:50.237908 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdc457cb7de65c18adf5ce38c84f1b6778e42494c73a6c925bd8b2184ad3f666\": container with ID starting with fdc457cb7de65c18adf5ce38c84f1b6778e42494c73a6c925bd8b2184ad3f666 not found: ID does not exist" containerID="fdc457cb7de65c18adf5ce38c84f1b6778e42494c73a6c925bd8b2184ad3f666" Nov 04 07:42:50.237983 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.237933 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdc457cb7de65c18adf5ce38c84f1b6778e42494c73a6c925bd8b2184ad3f666"} err="failed to get container status \"fdc457cb7de65c18adf5ce38c84f1b6778e42494c73a6c925bd8b2184ad3f666\": rpc error: code = NotFound desc = could not find container \"fdc457cb7de65c18adf5ce38c84f1b6778e42494c73a6c925bd8b2184ad3f666\": container with ID starting with fdc457cb7de65c18adf5ce38c84f1b6778e42494c73a6c925bd8b2184ad3f666 not found: ID does not exist" Nov 04 07:42:50.243945 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.243922 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc"] Nov 04 07:42:50.247956 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.247935 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-mtfhc"] Nov 04 07:42:50.267824 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.267786 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx"] Nov 04 07:42:50.269342 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.269323 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-cwkrx"] Nov 04 07:42:50.283601 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.283577 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z"] Nov 04 07:42:50.291404 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.291383 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-df3be8-openshift-default-854766f5d9-hvb9z"] Nov 04 07:42:50.587041 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.586959 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:50.814392 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.814357 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2b293d12-35ef-47df-9c77-27f734441df4" path="/var/lib/kubelet/pods/2b293d12-35ef-47df-9c77-27f734441df4/volumes" Nov 04 07:42:50.814840 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.814825 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="99d41a69-904b-44b9-9a38-253d859dac34" path="/var/lib/kubelet/pods/99d41a69-904b-44b9-9a38-253d859dac34/volumes" Nov 04 07:42:50.815217 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.815205 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a0faffc2-896c-4420-b382-47ecdcb49abb" path="/var/lib/kubelet/pods/a0faffc2-896c-4420-b382-47ecdcb49abb/volumes" Nov 04 07:42:50.815897 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:50.815883 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" path="/var/lib/kubelet/pods/c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8/volumes" Nov 04 07:42:54.345730 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:54.345700 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh"] Nov 04 07:42:54.346146 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:54.345899 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" podUID="20dfe3ac-721e-49c5-8d59-64cf1c9b506f" containerName="maas-api" containerID="cri-o://7a066357f9488207d27d9198b35c6cafb838b332645117a3994cfc3579aaa272" gracePeriod=30 Nov 04 07:42:54.372690 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:54.372649 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" podUID="20dfe3ac-721e-49c5-8d59-64cf1c9b506f" containerName="maas-api" probeResult="failure" output="Get \"https://10.133.0.46:8443/health\": dial tcp 10.133.0.46:8443: connect: connection refused" Nov 04 07:42:54.421263 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:54.421221 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:54.634057 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:54.634033 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:42:54.723837 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:54.723803 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-metrics-certs\") pod \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\" (UID: \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\") " Nov 04 07:42:54.723837 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:54.723850 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-openshift-service-ca\") pod \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\" (UID: \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\") " Nov 04 07:42:54.724084 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:54.723911 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-kube-api-access-npz8v\" (UniqueName: \"kubernetes.io/projected/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-kube-api-access-npz8v\") pod \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\" (UID: \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\") " Nov 04 07:42:54.724084 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:54.723955 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-maas-api-tls\") pod \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\" (UID: \"20dfe3ac-721e-49c5-8d59-64cf1c9b506f\") " Nov 04 07:42:54.724303 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:54.724277 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-openshift-service-ca" pod "20dfe3ac-721e-49c5-8d59-64cf1c9b506f" (UID: "20dfe3ac-721e-49c5-8d59-64cf1c9b506f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:42:54.725992 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:54.725964 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-metrics-certs" pod "20dfe3ac-721e-49c5-8d59-64cf1c9b506f" (UID: "20dfe3ac-721e-49c5-8d59-64cf1c9b506f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:42:54.725992 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:54.725974 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-kube-api-access-npz8v" pod "20dfe3ac-721e-49c5-8d59-64cf1c9b506f" (UID: "20dfe3ac-721e-49c5-8d59-64cf1c9b506f"). InnerVolumeSpecName "kube-api-access-npz8v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:42:54.726113 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:54.726021 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-maas-api-tls" pod "20dfe3ac-721e-49c5-8d59-64cf1c9b506f" (UID: "20dfe3ac-721e-49c5-8d59-64cf1c9b506f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 07:42:54.824914 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:54.824888 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npz8v\" (UniqueName: \"kubernetes.io/projected/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-kube-api-access-npz8v\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:54.824914 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:54.824915 5385 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-maas-api-tls\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:54.825042 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:54.824924 5385 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-metrics-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:54.825042 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:54.824934 5385 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/20dfe3ac-721e-49c5-8d59-64cf1c9b506f-openshift-service-ca\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:42:55.226338 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:55.226302 5385 generic.go:356] "Generic (PLEG): container finished" podID="20dfe3ac-721e-49c5-8d59-64cf1c9b506f" containerID="7a066357f9488207d27d9198b35c6cafb838b332645117a3994cfc3579aaa272" exitCode=0 Nov 04 07:42:55.226487 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:55.226365 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" Nov 04 07:42:55.226487 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:55.226382 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" event={"ID":"20dfe3ac-721e-49c5-8d59-64cf1c9b506f","Type":"ContainerDied","Data":"7a066357f9488207d27d9198b35c6cafb838b332645117a3994cfc3579aaa272"} Nov 04 07:42:55.226487 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:55.226438 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh" event={"ID":"20dfe3ac-721e-49c5-8d59-64cf1c9b506f","Type":"ContainerDied","Data":"4cdcfecd1dc2d93e82b73eb89b1c490ce2ce0ea7555f1dcc5b0592b6dc4147b8"} Nov 04 07:42:55.226487 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:55.226460 5385 scope.go:122] "RemoveContainer" containerID="7a066357f9488207d27d9198b35c6cafb838b332645117a3994cfc3579aaa272" Nov 04 07:42:55.235301 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:55.235281 5385 scope.go:122] "RemoveContainer" containerID="7a066357f9488207d27d9198b35c6cafb838b332645117a3994cfc3579aaa272" Nov 04 07:42:55.235592 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:42:55.235574 5385 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a066357f9488207d27d9198b35c6cafb838b332645117a3994cfc3579aaa272\": container with ID starting with 7a066357f9488207d27d9198b35c6cafb838b332645117a3994cfc3579aaa272 not found: ID does not exist" containerID="7a066357f9488207d27d9198b35c6cafb838b332645117a3994cfc3579aaa272" Nov 04 07:42:55.235641 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:55.235601 5385 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a066357f9488207d27d9198b35c6cafb838b332645117a3994cfc3579aaa272"} err="failed to get container status \"7a066357f9488207d27d9198b35c6cafb838b332645117a3994cfc3579aaa272\": rpc error: code = NotFound desc = could not find container \"7a066357f9488207d27d9198b35c6cafb838b332645117a3994cfc3579aaa272\": container with ID starting with 7a066357f9488207d27d9198b35c6cafb838b332645117a3994cfc3579aaa272 not found: ID does not exist" Nov 04 07:42:55.244182 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:55.244159 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh"] Nov 04 07:42:55.248599 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:55.248572 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1f6ce4-6b4d7dd66d-rj5jh"] Nov 04 07:42:56.813545 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:56.813496 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20dfe3ac-721e-49c5-8d59-64cf1c9b506f" path="/var/lib/kubelet/pods/20dfe3ac-721e-49c5-8d59-64cf1c9b506f/volumes" Nov 04 07:42:56.991023 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:56.990985 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:42:57.159996 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:42:57.159960 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:43:00.875042 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:43:00.875004 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:43:12.559998 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:12.559948 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:43:12.663303 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:12.663263 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:43:14.039802 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:14.039763 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:43:14.798144 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:14.798109 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:43:15.138478 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:15.138445 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4"] Nov 04 07:43:15.138929 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:15.138772 5385 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" podUID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" containerName="istio-proxy" containerID="cri-o://d8b999597a48cd5bac0e08c50029352c92d594bb438f34450efe2f289a6fe8bd" gracePeriod=30 Nov 04 07:43:15.182849 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:15.182751 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:43:15.307550 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:15.307487 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:43:16.668988 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:16.668908 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:43:17.909869 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:17.909834 5385 patch_prober.go:29] interesting pod/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Nov 04 07:43:17.910250 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:17.909903 5385 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" podUID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Nov 04 07:43:18.730905 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:18.730875 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-clnnz_a1444ffa-7440-4c39-82d0-e2fabf7cf4e3/manager/0.log" Nov 04 07:43:18.907308 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:18.907280 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-zzdxv_a23d550f-ddb6-4615-befd-cdaf1ed4d185/manager/0.log" Nov 04 07:43:19.345969 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:19.345937 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-779b45f986-ng2mv_0c4eee75-8211-4d94-80de-20aeab2fbbac/manager/0.log" Nov 04 07:43:19.977697 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:19.977665 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-zgbmm_44f8bc78-1413-4eba-a680-e22ff3b94f3b/proxy/0.log" Nov 04 07:43:20.101195 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.101142 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:43:20.217252 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.217201 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:43:20.345215 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.345184 5385 generic.go:356] "Generic (PLEG): container finished" podID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" containerID="d8b999597a48cd5bac0e08c50029352c92d594bb438f34450efe2f289a6fe8bd" exitCode=0 Nov 04 07:43:20.345365 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.345244 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" event={"ID":"d990bf05-77a4-4ebd-a301-c55f5bb7d526","Type":"ContainerDied","Data":"d8b999597a48cd5bac0e08c50029352c92d594bb438f34450efe2f289a6fe8bd"} Nov 04 07:43:20.345365 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.345291 5385 scope.go:122] "RemoveContainer" containerID="cb66ca294a82a3c4a5dd785ba1077f0b517b1f192067e0556b1896fe8269b2e7" Nov 04 07:43:20.395494 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.395472 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:43:20.464048 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464013 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-envoy\") pod \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " Nov 04 07:43:20.464221 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464063 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-credential-socket\") pod \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " Nov 04 07:43:20.464221 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464124 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-data\") pod \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " Nov 04 07:43:20.464221 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464171 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-podinfo\") pod \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " Nov 04 07:43:20.464221 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464202 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-workload-certs\") pod \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " Nov 04 07:43:20.464434 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464232 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-workload-socket\") pod \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " Nov 04 07:43:20.464434 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464272 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-token\" (UniqueName: \"kubernetes.io/projected/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-token\") pod \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " Nov 04 07:43:20.464434 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464308 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d990bf05-77a4-4ebd-a301-c55f5bb7d526-kube-api-access-bkq74\" (UniqueName: \"kubernetes.io/projected/d990bf05-77a4-4ebd-a301-c55f5bb7d526-kube-api-access-bkq74\") pod \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " Nov 04 07:43:20.464434 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464380 5385 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istiod-ca-cert\") pod \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\" (UID: \"d990bf05-77a4-4ebd-a301-c55f5bb7d526\") " Nov 04 07:43:20.464434 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464387 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-credential-socket" pod "d990bf05-77a4-4ebd-a301-c55f5bb7d526" (UID: "d990bf05-77a4-4ebd-a301-c55f5bb7d526"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:43:20.464721 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464554 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-data" pod "d990bf05-77a4-4ebd-a301-c55f5bb7d526" (UID: "d990bf05-77a4-4ebd-a301-c55f5bb7d526"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:43:20.464721 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464575 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-workload-certs" pod "d990bf05-77a4-4ebd-a301-c55f5bb7d526" (UID: "d990bf05-77a4-4ebd-a301-c55f5bb7d526"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:43:20.464721 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464593 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-workload-socket" pod "d990bf05-77a4-4ebd-a301-c55f5bb7d526" (UID: "d990bf05-77a4-4ebd-a301-c55f5bb7d526"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:43:20.464881 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464757 5385 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-credential-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:43:20.464881 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464779 5385 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-data\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:43:20.464881 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464794 5385 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-workload-certs\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:43:20.464881 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464809 5385 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-workload-socket\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:43:20.465066 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.464902 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istiod-ca-cert" pod "d990bf05-77a4-4ebd-a301-c55f5bb7d526" (UID: "d990bf05-77a4-4ebd-a301-c55f5bb7d526"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 07:43:20.466321 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.466292 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-envoy" pod "d990bf05-77a4-4ebd-a301-c55f5bb7d526" (UID: "d990bf05-77a4-4ebd-a301-c55f5bb7d526"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 07:43:20.466453 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.466435 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-podinfo" pod "d990bf05-77a4-4ebd-a301-c55f5bb7d526" (UID: "d990bf05-77a4-4ebd-a301-c55f5bb7d526"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 07:43:20.466499 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.466471 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-token" pod "d990bf05-77a4-4ebd-a301-c55f5bb7d526" (UID: "d990bf05-77a4-4ebd-a301-c55f5bb7d526"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:43:20.466602 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.466586 5385 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d990bf05-77a4-4ebd-a301-c55f5bb7d526-kube-api-access-bkq74" pod "d990bf05-77a4-4ebd-a301-c55f5bb7d526" (UID: "d990bf05-77a4-4ebd-a301-c55f5bb7d526"). InnerVolumeSpecName "kube-api-access-bkq74". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 07:43:20.565970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.565901 5385 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istiod-ca-cert\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:43:20.565970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.565927 5385 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-envoy\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:43:20.565970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.565936 5385 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-podinfo\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:43:20.565970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.565944 5385 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d990bf05-77a4-4ebd-a301-c55f5bb7d526-istio-token\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:43:20.565970 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:20.565953 5385 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkq74\" (UniqueName: \"kubernetes.io/projected/d990bf05-77a4-4ebd-a301-c55f5bb7d526-kube-api-access-bkq74\") on node \"ip-10-0-131-6.us-east-2.compute.internal\" DevicePath \"\"" Nov 04 07:43:21.351990 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:21.351953 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" event={"ID":"d990bf05-77a4-4ebd-a301-c55f5bb7d526","Type":"ContainerDied","Data":"21ca19cdafc2002cb1f520532f15c1dce06eec948200cf78956ee30ee01f2cf5"} Nov 04 07:43:21.351990 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:21.351986 5385 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4" Nov 04 07:43:21.352201 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:21.352023 5385 scope.go:122] "RemoveContainer" containerID="d8b999597a48cd5bac0e08c50029352c92d594bb438f34450efe2f289a6fe8bd" Nov 04 07:43:21.371741 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:21.371714 5385 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4"] Nov 04 07:43:21.381419 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:21.381392 5385 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-1f6ce4-openshift-default-774f8ff68c-txnf4"] Nov 04 07:43:22.811447 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:22.811413 5385 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" path="/var/lib/kubelet/pods/d990bf05-77a4-4ebd-a301-c55f5bb7d526/volumes" Nov 04 07:43:30.874123 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:43:30.874084 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:43:35.743645 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:35.743614 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-clnnz_a1444ffa-7440-4c39-82d0-e2fabf7cf4e3/manager/0.log" Nov 04 07:43:35.931639 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:35.931607 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-zzdxv_a23d550f-ddb6-4615-befd-cdaf1ed4d185/manager/0.log" Nov 04 07:43:36.391610 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:36.391580 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-779b45f986-ng2mv_0c4eee75-8211-4d94-80de-20aeab2fbbac/manager/0.log" Nov 04 07:43:37.012117 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:37.012084 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-zgbmm_44f8bc78-1413-4eba-a680-e22ff3b94f3b/proxy/0.log" Nov 04 07:43:41.303753 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:41.303724 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-n89nj_2b751fab-8c5e-4b42-bc7f-6a046157d11c/postgres/0.log" Nov 04 07:43:46.059543 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:46.059485 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-d7xzv_26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3/discovery/0.log" Nov 04 07:43:46.243010 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:46.242979 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589fcc7ff4-dcwx7_ebcd9bc5-c463-4550-820d-6648936b12a3/kube-auth-proxy/0.log" Nov 04 07:43:46.962744 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:46.962712 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-7wfxn_c3c860e0-8952-4f74-acd1-803a8a6acc85/istio-proxy/0.log" Nov 04 07:43:47.135392 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:47.135360 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-bx8vz_e734faeb-28a7-4aa2-86cc-dc9c24cf5c34/istio-proxy/0.log" Nov 04 07:43:47.492847 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:47.492810 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-lhm7v_ae22c880-563f-4c2c-9461-b1e185adf358/istio-proxy/0.log" Nov 04 07:43:48.618534 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:48.618483 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-7ppvl_27d5958b-d699-4846-b674-3bf2ef6d25ca/payload-pre-processing/0.log" Nov 04 07:43:48.794577 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:48.794543 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wzjpf_4522cb36-2dcc-483d-bf1e-42ffd1029cee/payload-processing/0.log" Nov 04 07:43:50.134845 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:50.134814 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-cbd4f5586-z9dmc_46148a51-0296-47b0-abce-48b086c3d94e/main/0.log" Nov 04 07:43:50.488891 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:50.488811 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-4rscf_526529db-d790-40e3-9983-bc2f8473a034/main/0.log" Nov 04 07:43:51.042933 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:51.042902 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-q9dnj_34c2083c-6a37-460e-b6bd-8510768d9e4e/main/0.log" Nov 04 07:43:51.223812 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:51.223782 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-84b4ff656c-d49z2_092adbcf-64c3-4ac6-80a7-9ce2d00e675a/main/0.log" Nov 04 07:43:56.982846 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:43:56.982809 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:44:00.874824 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:44:00.874790 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:44:01.633991 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.633951 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zbgbj/must-gather-f6jtz"] Nov 04 07:44:01.634422 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634399 5385 cpu_manager.go:414] "containerMap: removing container" podUID="2b293d12-35ef-47df-9c77-27f734441df4" containerName="istio-proxy" Nov 04 07:44:01.634422 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634421 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b293d12-35ef-47df-9c77-27f734441df4" containerName="istio-proxy" Nov 04 07:44:01.634627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634439 5385 cpu_manager.go:414] "containerMap: removing container" podUID="99d41a69-904b-44b9-9a38-253d859dac34" containerName="istio-proxy" Nov 04 07:44:01.634627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634445 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="99d41a69-904b-44b9-9a38-253d859dac34" containerName="istio-proxy" Nov 04 07:44:01.634627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634467 5385 cpu_manager.go:414] "containerMap: removing container" podUID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" containerName="istio-proxy" Nov 04 07:44:01.634627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634475 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" containerName="istio-proxy" Nov 04 07:44:01.634627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634494 5385 cpu_manager.go:414] "containerMap: removing container" podUID="2b293d12-35ef-47df-9c77-27f734441df4" containerName="istio-proxy" Nov 04 07:44:01.634627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634499 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2b293d12-35ef-47df-9c77-27f734441df4" containerName="istio-proxy" Nov 04 07:44:01.634627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634526 5385 cpu_manager.go:414] "containerMap: removing container" podUID="a0faffc2-896c-4420-b382-47ecdcb49abb" containerName="istio-proxy" Nov 04 07:44:01.634627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634531 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a0faffc2-896c-4420-b382-47ecdcb49abb" containerName="istio-proxy" Nov 04 07:44:01.634627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634542 5385 cpu_manager.go:414] "containerMap: removing container" podUID="c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" containerName="istio-proxy" Nov 04 07:44:01.634627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634548 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" containerName="istio-proxy" Nov 04 07:44:01.634627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634554 5385 cpu_manager.go:414] "containerMap: removing container" podUID="63ab0edc-d9ba-486d-bfc0-7060678daea2" containerName="istio-proxy" Nov 04 07:44:01.634627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634559 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63ab0edc-d9ba-486d-bfc0-7060678daea2" containerName="istio-proxy" Nov 04 07:44:01.634627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634582 5385 cpu_manager.go:414] "containerMap: removing container" podUID="20dfe3ac-721e-49c5-8d59-64cf1c9b506f" containerName="maas-api" Nov 04 07:44:01.634627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634586 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20dfe3ac-721e-49c5-8d59-64cf1c9b506f" containerName="maas-api" Nov 04 07:44:01.634627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634596 5385 cpu_manager.go:414] "containerMap: removing container" podUID="63ab0edc-d9ba-486d-bfc0-7060678daea2" containerName="istio-proxy" Nov 04 07:44:01.634627 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634601 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63ab0edc-d9ba-486d-bfc0-7060678daea2" containerName="istio-proxy" Nov 04 07:44:01.635154 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634660 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" containerName="istio-proxy" Nov 04 07:44:01.635154 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634668 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="20dfe3ac-721e-49c5-8d59-64cf1c9b506f" containerName="maas-api" Nov 04 07:44:01.635154 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634675 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="63ab0edc-d9ba-486d-bfc0-7060678daea2" containerName="istio-proxy" Nov 04 07:44:01.635154 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634681 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b293d12-35ef-47df-9c77-27f734441df4" containerName="istio-proxy" Nov 04 07:44:01.635154 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634686 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="99d41a69-904b-44b9-9a38-253d859dac34" containerName="istio-proxy" Nov 04 07:44:01.635154 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634693 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" containerName="istio-proxy" Nov 04 07:44:01.635154 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634699 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="2b293d12-35ef-47df-9c77-27f734441df4" containerName="istio-proxy" Nov 04 07:44:01.635154 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634704 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="a0faffc2-896c-4420-b382-47ecdcb49abb" containerName="istio-proxy" Nov 04 07:44:01.635154 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634710 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="c52ab653-d09f-4e9b-bfa3-b37acc4ea0d8" containerName="istio-proxy" Nov 04 07:44:01.635154 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634782 5385 cpu_manager.go:414] "containerMap: removing container" podUID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" containerName="istio-proxy" Nov 04 07:44:01.635154 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634787 5385 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d990bf05-77a4-4ebd-a301-c55f5bb7d526" containerName="istio-proxy" Nov 04 07:44:01.635154 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.634840 5385 memory_manager.go:365] "RemoveStaleState removing state" podUID="63ab0edc-d9ba-486d-bfc0-7060678daea2" containerName="istio-proxy" Nov 04 07:44:01.637921 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.637904 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zbgbj/must-gather-f6jtz" Nov 04 07:44:01.647782 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.647757 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zbgbj\"/\"default-dockercfg-8trrv\"" Nov 04 07:44:01.647782 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.647778 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zbgbj\"/\"openshift-service-ca.crt\"" Nov 04 07:44:01.647926 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.647810 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zbgbj\"/\"kube-root-ca.crt\"" Nov 04 07:44:01.654233 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.654210 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zbgbj/must-gather-f6jtz"] Nov 04 07:44:01.685194 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.685164 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:44:01.735877 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.735845 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-299rc\" (UniqueName: \"kubernetes.io/projected/0c063ee3-f512-447a-883c-8db551785c46-kube-api-access-299rc\") pod \"must-gather-f6jtz\" (UID: \"0c063ee3-f512-447a-883c-8db551785c46\") " pod="openshift-must-gather-zbgbj/must-gather-f6jtz" Nov 04 07:44:01.736007 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.735915 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0c063ee3-f512-447a-883c-8db551785c46-must-gather-output\") pod \"must-gather-f6jtz\" (UID: \"0c063ee3-f512-447a-883c-8db551785c46\") " pod="openshift-must-gather-zbgbj/must-gather-f6jtz" Nov 04 07:44:01.836980 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.836946 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0c063ee3-f512-447a-883c-8db551785c46-must-gather-output\") pod \"must-gather-f6jtz\" (UID: \"0c063ee3-f512-447a-883c-8db551785c46\") " pod="openshift-must-gather-zbgbj/must-gather-f6jtz" Nov 04 07:44:01.837149 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.837096 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-299rc\" (UniqueName: \"kubernetes.io/projected/0c063ee3-f512-447a-883c-8db551785c46-kube-api-access-299rc\") pod \"must-gather-f6jtz\" (UID: \"0c063ee3-f512-447a-883c-8db551785c46\") " pod="openshift-must-gather-zbgbj/must-gather-f6jtz" Nov 04 07:44:01.837340 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.837318 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0c063ee3-f512-447a-883c-8db551785c46-must-gather-output\") pod \"must-gather-f6jtz\" (UID: \"0c063ee3-f512-447a-883c-8db551785c46\") " pod="openshift-must-gather-zbgbj/must-gather-f6jtz" Nov 04 07:44:01.838358 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.838332 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:44:01.849690 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.849668 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-299rc\" (UniqueName: \"kubernetes.io/projected/0c063ee3-f512-447a-883c-8db551785c46-kube-api-access-299rc\") pod \"must-gather-f6jtz\" (UID: \"0c063ee3-f512-447a-883c-8db551785c46\") " pod="openshift-must-gather-zbgbj/must-gather-f6jtz" Nov 04 07:44:01.958163 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:01.958089 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zbgbj/must-gather-f6jtz" Nov 04 07:44:02.255726 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:02.255702 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zbgbj/must-gather-f6jtz"] Nov 04 07:44:02.258209 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:44:02.258179 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c063ee3_f512_447a_883c_8db551785c46.slice/crio-d74e42f24efd98bcc96fda2c13945c3252ac1a71df9396feb527a8f1de73b87b WatchSource:0}: Error finding container d74e42f24efd98bcc96fda2c13945c3252ac1a71df9396feb527a8f1de73b87b: Status 404 returned error can't find the container with id d74e42f24efd98bcc96fda2c13945c3252ac1a71df9396feb527a8f1de73b87b Nov 04 07:44:02.260013 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:02.259996 5385 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 07:44:02.539555 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:02.539452 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zbgbj/must-gather-f6jtz" event={"ID":"0c063ee3-f512-447a-883c-8db551785c46","Type":"ContainerStarted","Data":"d74e42f24efd98bcc96fda2c13945c3252ac1a71df9396feb527a8f1de73b87b"} Nov 04 07:44:05.566740 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:05.566711 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-m7gt8_fa9b645b-69d5-47a7-8457-409364209023/global-pull-secret-syncer/0.log" Nov 04 07:44:05.851142 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:05.851041 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" Nov 04 07:44:06.071469 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:06.071441 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-6.us-east-2.compute.internal_b7ad388a799a062d464456d3dcd4ae6a/haproxy/0.log" Nov 04 07:44:08.574193 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:08.574160 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zbgbj/must-gather-f6jtz" event={"ID":"0c063ee3-f512-447a-883c-8db551785c46","Type":"ContainerStarted","Data":"e43bbb2cfd72ab1debf4ad9b705f910521736e16a12eaa2fdb1a1c75bd5491ed"} Nov 04 07:44:08.574659 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:08.574198 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zbgbj/must-gather-f6jtz" event={"ID":"0c063ee3-f512-447a-883c-8db551785c46","Type":"ContainerStarted","Data":"396a830b245f98bdcb6a88fe45589f2a75aa9232b71122cce407679df985c323"} Nov 04 07:44:08.596021 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:08.595972 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-zbgbj/must-gather-f6jtz" podStartSLOduration=2.083744187 podStartE2EDuration="7.595959588s" totalImagesPullingTime="5.512215401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 07:44:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:44:08.594109807 +0000 UTC m=+1683.928049393" watchObservedRunningTime="2026-11-04 07:44:08.595959588 +0000 UTC m=+1683.929899171" Nov 04 07:44:09.155442 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:09.155413 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-clnnz_a1444ffa-7440-4c39-82d0-e2fabf7cf4e3/manager/0.log" Nov 04 07:44:09.222259 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:09.222226 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-zzdxv_a23d550f-ddb6-4615-befd-cdaf1ed4d185/manager/0.log" Nov 04 07:44:09.540039 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:09.539972 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-779b45f986-ng2mv_0c4eee75-8211-4d94-80de-20aeab2fbbac/manager/0.log" Nov 04 07:44:10.155317 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:10.155289 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-zgbmm_44f8bc78-1413-4eba-a680-e22ff3b94f3b/proxy/0.log" Nov 04 07:44:11.127249 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:11.127221 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-clnnz_a1444ffa-7440-4c39-82d0-e2fabf7cf4e3/manager/0.log" Nov 04 07:44:11.172968 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:11.172934 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-zzdxv_a23d550f-ddb6-4615-befd-cdaf1ed4d185/manager/0.log" Nov 04 07:44:11.427828 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:11.427758 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-779b45f986-ng2mv_0c4eee75-8211-4d94-80de-20aeab2fbbac/manager/0.log" Nov 04 07:44:12.020588 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:12.020555 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-zgbmm_44f8bc78-1413-4eba-a680-e22ff3b94f3b/proxy/0.log" Nov 04 07:44:12.956699 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:12.956671 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-clnnz_a1444ffa-7440-4c39-82d0-e2fabf7cf4e3/manager/0.log" Nov 04 07:44:13.052631 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:13.052603 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-zzdxv_a23d550f-ddb6-4615-befd-cdaf1ed4d185/manager/0.log" Nov 04 07:44:13.354160 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:13.354086 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-779b45f986-ng2mv_0c4eee75-8211-4d94-80de-20aeab2fbbac/manager/0.log" Nov 04 07:44:13.974919 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:13.974882 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-zgbmm_44f8bc78-1413-4eba-a680-e22ff3b94f3b/proxy/0.log" Nov 04 07:44:14.735665 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:14.735631 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58fe8ba7-05e1-42f3-8354-aabdf391d0e4/alertmanager/0.log" Nov 04 07:44:14.777440 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:14.777401 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58fe8ba7-05e1-42f3-8354-aabdf391d0e4/config-reloader/0.log" Nov 04 07:44:14.813098 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:14.813069 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58fe8ba7-05e1-42f3-8354-aabdf391d0e4/kube-rbac-proxy-web/0.log" Nov 04 07:44:14.866353 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:14.866320 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58fe8ba7-05e1-42f3-8354-aabdf391d0e4/kube-rbac-proxy/0.log" Nov 04 07:44:14.922960 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:14.922930 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58fe8ba7-05e1-42f3-8354-aabdf391d0e4/kube-rbac-proxy-metric/0.log" Nov 04 07:44:14.956265 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:14.956239 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58fe8ba7-05e1-42f3-8354-aabdf391d0e4/prom-label-proxy/0.log" Nov 04 07:44:14.996558 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:14.996475 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_58fe8ba7-05e1-42f3-8354-aabdf391d0e4/init-config-reloader/0.log" Nov 04 07:44:15.438566 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.438499 5385 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r"] Nov 04 07:44:15.448025 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.447994 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:15.451644 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.451614 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r"] Nov 04 07:44:15.522082 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.522056 5385 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-zs2zc\"/\"default-dockercfg-6mkpg\"" Nov 04 07:44:15.522713 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.522689 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zs2zc\"/\"openshift-service-ca.crt\"" Nov 04 07:44:15.522840 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.522688 5385 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-zs2zc\"/\"kube-root-ca.crt\"" Nov 04 07:44:15.535852 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.535820 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ktbsj_1488becb-97bc-43e6-b85c-6c9c62dfe903/node-exporter/0.log" Nov 04 07:44:15.568563 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.568528 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0783cbec-7f03-4785-a35e-fbb09887c2b0-podres\") pod \"perf-node-gather-daemonset-f5j9r\" (UID: \"0783cbec-7f03-4785-a35e-fbb09887c2b0\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:15.568707 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.568588 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0783cbec-7f03-4785-a35e-fbb09887c2b0-lib-modules\") pod \"perf-node-gather-daemonset-f5j9r\" (UID: \"0783cbec-7f03-4785-a35e-fbb09887c2b0\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:15.568707 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.568668 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95jsq\" (UniqueName: \"kubernetes.io/projected/0783cbec-7f03-4785-a35e-fbb09887c2b0-kube-api-access-95jsq\") pod \"perf-node-gather-daemonset-f5j9r\" (UID: \"0783cbec-7f03-4785-a35e-fbb09887c2b0\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:15.568822 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.568779 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0783cbec-7f03-4785-a35e-fbb09887c2b0-sys\") pod \"perf-node-gather-daemonset-f5j9r\" (UID: \"0783cbec-7f03-4785-a35e-fbb09887c2b0\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:15.568864 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.568840 5385 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0783cbec-7f03-4785-a35e-fbb09887c2b0-proc\") pod \"perf-node-gather-daemonset-f5j9r\" (UID: \"0783cbec-7f03-4785-a35e-fbb09887c2b0\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:15.593806 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.593788 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ktbsj_1488becb-97bc-43e6-b85c-6c9c62dfe903/kube-rbac-proxy/0.log" Nov 04 07:44:15.652918 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.652895 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ktbsj_1488becb-97bc-43e6-b85c-6c9c62dfe903/init-textfile/0.log" Nov 04 07:44:15.669320 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.669291 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95jsq\" (UniqueName: \"kubernetes.io/projected/0783cbec-7f03-4785-a35e-fbb09887c2b0-kube-api-access-95jsq\") pod \"perf-node-gather-daemonset-f5j9r\" (UID: \"0783cbec-7f03-4785-a35e-fbb09887c2b0\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:15.669534 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.669333 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0783cbec-7f03-4785-a35e-fbb09887c2b0-sys\") pod \"perf-node-gather-daemonset-f5j9r\" (UID: \"0783cbec-7f03-4785-a35e-fbb09887c2b0\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:15.669534 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.669363 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0783cbec-7f03-4785-a35e-fbb09887c2b0-proc\") pod \"perf-node-gather-daemonset-f5j9r\" (UID: \"0783cbec-7f03-4785-a35e-fbb09887c2b0\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:15.669534 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.669405 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0783cbec-7f03-4785-a35e-fbb09887c2b0-podres\") pod \"perf-node-gather-daemonset-f5j9r\" (UID: \"0783cbec-7f03-4785-a35e-fbb09887c2b0\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:15.669534 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.669439 5385 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0783cbec-7f03-4785-a35e-fbb09887c2b0-lib-modules\") pod \"perf-node-gather-daemonset-f5j9r\" (UID: \"0783cbec-7f03-4785-a35e-fbb09887c2b0\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:15.669534 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.669450 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/0783cbec-7f03-4785-a35e-fbb09887c2b0-proc\") pod \"perf-node-gather-daemonset-f5j9r\" (UID: \"0783cbec-7f03-4785-a35e-fbb09887c2b0\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:15.669534 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.669450 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0783cbec-7f03-4785-a35e-fbb09887c2b0-sys\") pod \"perf-node-gather-daemonset-f5j9r\" (UID: \"0783cbec-7f03-4785-a35e-fbb09887c2b0\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:15.669963 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.669558 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/0783cbec-7f03-4785-a35e-fbb09887c2b0-podres\") pod \"perf-node-gather-daemonset-f5j9r\" (UID: \"0783cbec-7f03-4785-a35e-fbb09887c2b0\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:15.669963 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.669604 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0783cbec-7f03-4785-a35e-fbb09887c2b0-lib-modules\") pod \"perf-node-gather-daemonset-f5j9r\" (UID: \"0783cbec-7f03-4785-a35e-fbb09887c2b0\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:15.685530 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.685492 5385 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95jsq\" (UniqueName: \"kubernetes.io/projected/0783cbec-7f03-4785-a35e-fbb09887c2b0-kube-api-access-95jsq\") pod \"perf-node-gather-daemonset-f5j9r\" (UID: \"0783cbec-7f03-4785-a35e-fbb09887c2b0\") " pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:15.765037 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:15.764940 5385 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:16.065363 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:16.065252 5385 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r"] Nov 04 07:44:16.068934 ip-10-0-131-6 kubenswrapper[5385]: W1104 07:44:16.068899 5385 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod0783cbec_7f03_4785_a35e_fbb09887c2b0.slice/crio-d13da292fcd72210fd4ae590c7774e5354545f260fff2d7c689013b0d68fef07 WatchSource:0}: Error finding container d13da292fcd72210fd4ae590c7774e5354545f260fff2d7c689013b0d68fef07: Status 404 returned error can't find the container with id d13da292fcd72210fd4ae590c7774e5354545f260fff2d7c689013b0d68fef07 Nov 04 07:44:16.352796 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:16.352699 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-sh7s5_51538327-34ef-409b-b76a-6963b7254eec/prometheus-operator/0.log" Nov 04 07:44:16.415049 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:16.415015 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-sh7s5_51538327-34ef-409b-b76a-6963b7254eec/kube-rbac-proxy/0.log" Nov 04 07:44:16.461980 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:16.461946 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-d7xzv_26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3/discovery/0.log" Nov 04 07:44:16.492756 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:16.492733 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589fcc7ff4-dcwx7_ebcd9bc5-c463-4550-820d-6648936b12a3/kube-auth-proxy/0.log" Nov 04 07:44:16.569750 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:16.569718 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6985d69b67-82vf8_434549d4-e06f-4fb9-a903-9d53cb4646d5/telemeter-client/0.log" Nov 04 07:44:16.617012 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:16.616933 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" event={"ID":"0783cbec-7f03-4785-a35e-fbb09887c2b0","Type":"ContainerStarted","Data":"3e77710ec6f37b69b998a31830ad9af2c75b4e51f43ef0f33b7926b844eaf891"} Nov 04 07:44:16.617160 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:16.617023 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:16.617160 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:16.617044 5385 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" event={"ID":"0783cbec-7f03-4785-a35e-fbb09887c2b0","Type":"ContainerStarted","Data":"d13da292fcd72210fd4ae590c7774e5354545f260fff2d7c689013b0d68fef07"} Nov 04 07:44:16.617913 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:16.617892 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6985d69b67-82vf8_434549d4-e06f-4fb9-a903-9d53cb4646d5/reload/0.log" Nov 04 07:44:16.625518 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:16.625486 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-7wfxn_c3c860e0-8952-4f74-acd1-803a8a6acc85/istio-proxy/0.log" Nov 04 07:44:16.654753 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:16.654729 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-bx8vz_e734faeb-28a7-4aa2-86cc-dc9c24cf5c34/istio-proxy/0.log" Nov 04 07:44:16.663197 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:16.663176 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6985d69b67-82vf8_434549d4-e06f-4fb9-a903-9d53cb4646d5/kube-rbac-proxy/0.log" Nov 04 07:44:16.719522 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:16.719487 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-lhm7v_ae22c880-563f-4c2c-9461-b1e185adf358/istio-proxy/0.log" Nov 04 07:44:16.819495 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:16.819455 5385 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2s2xx_ae126ffc-5714-4d59-b4ad-6f2baa0cf1a9/konnectivity-agent/0.log" limit=1024 Nov 04 07:44:16.900603 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:16.900576 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-7ppvl_27d5958b-d699-4846-b674-3bf2ef6d25ca/payload-pre-processing/0.log" Nov 04 07:44:16.921054 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:16.921028 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wzjpf_4522cb36-2dcc-483d-bf1e-42ffd1029cee/payload-processing/0.log" Nov 04 07:44:19.710918 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:19.710887 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-78c87f586f-6twc6_5ef794e0-f534-4536-b524-f40f82b32980/console/0.log" Nov 04 07:44:22.071665 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:22.071634 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qpt88_508fee92-08e3-49ab-8b16-3add538b11eb/dns/0.log" Nov 04 07:44:22.108543 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:22.108486 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qpt88_508fee92-08e3-49ab-8b16-3add538b11eb/kube-rbac-proxy/0.log" Nov 04 07:44:22.235982 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:22.235945 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-x6v9g_19b2ab56-a868-4570-b098-37c517da236a/dns-node-resolver/0.log" Nov 04 07:44:22.632177 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:22.632146 5385 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" Nov 04 07:44:22.669563 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:22.669500 5385 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-zs2zc/perf-node-gather-daemonset-f5j9r" podStartSLOduration=7.669485438 podStartE2EDuration="7.669485438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 07:44:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 07:44:16.644598812 +0000 UTC m=+1691.978538397" watchObservedRunningTime="2026-11-04 07:44:22.669485438 +0000 UTC m=+1698.003425021" Nov 04 07:44:23.107299 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:23.107222 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-sd2bs_f33f6c60-eff2-40c2-9350-ba10eaf5442d/node-ca/0.log" Nov 04 07:44:25.449323 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:25.449292 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-8f94f88d4-2xfwc_ce14b071-5b23-4026-ae18-122fc62ebee8/manager/0.log" Nov 04 07:44:25.466818 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:25.466792 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-8f94f88d4-2xfwc_ce14b071-5b23-4026-ae18-122fc62ebee8/copy-manifests/0.log" Nov 04 07:44:25.855302 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:25.855217 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-d7xzv_26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3/discovery/0.log" Nov 04 07:44:25.890222 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:25.890199 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589fcc7ff4-dcwx7_ebcd9bc5-c463-4550-820d-6648936b12a3/kube-auth-proxy/0.log" Nov 04 07:44:26.135047 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:26.135022 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-7wfxn_c3c860e0-8952-4f74-acd1-803a8a6acc85/istio-proxy/0.log" Nov 04 07:44:26.170167 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:26.169280 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-bx8vz_e734faeb-28a7-4aa2-86cc-dc9c24cf5c34/istio-proxy/0.log" Nov 04 07:44:26.259111 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:26.259083 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-lhm7v_ae22c880-563f-4c2c-9461-b1e185adf358/istio-proxy/0.log" Nov 04 07:44:26.336566 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:26.336535 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-clnnz_a1444ffa-7440-4c39-82d0-e2fabf7cf4e3/manager/0.log" Nov 04 07:44:26.376559 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:26.376530 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-module-controller-manager-7566cf5fc8-zzdxv_a23d550f-ddb6-4615-befd-cdaf1ed4d185/manager/0.log" Nov 04 07:44:26.543912 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:26.543842 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-7ppvl_27d5958b-d699-4846-b674-3bf2ef6d25ca/payload-pre-processing/0.log" Nov 04 07:44:26.638294 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:26.638266 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wzjpf_4522cb36-2dcc-483d-bf1e-42ffd1029cee/payload-processing/0.log" Nov 04 07:44:26.735724 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:26.735696 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-779b45f986-ng2mv_0c4eee75-8211-4d94-80de-20aeab2fbbac/manager/0.log" Nov 04 07:44:27.314100 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:27.314069 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-zgbmm_44f8bc78-1413-4eba-a680-e22ff3b94f3b/proxy/0.log" Nov 04 07:44:27.763300 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:27.763264 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-d7xzv_26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3/discovery/0.log" Nov 04 07:44:27.784609 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:27.784584 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589fcc7ff4-dcwx7_ebcd9bc5-c463-4550-820d-6648936b12a3/kube-auth-proxy/0.log" Nov 04 07:44:27.884053 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:27.884020 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-7wfxn_c3c860e0-8952-4f74-acd1-803a8a6acc85/istio-proxy/0.log" Nov 04 07:44:27.909003 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:27.908966 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-bx8vz_e734faeb-28a7-4aa2-86cc-dc9c24cf5c34/istio-proxy/0.log" Nov 04 07:44:27.963442 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:27.963407 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-lhm7v_ae22c880-563f-4c2c-9461-b1e185adf358/istio-proxy/0.log" Nov 04 07:44:28.087856 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:28.087785 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-7ppvl_27d5958b-d699-4846-b674-3bf2ef6d25ca/payload-pre-processing/0.log" Nov 04 07:44:28.120481 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:28.120452 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wzjpf_4522cb36-2dcc-483d-bf1e-42ffd1029cee/payload-processing/0.log" Nov 04 07:44:28.207265 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:28.207237 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pksw6_d085676f-b286-4834-9e62-039f2b6e7f72/serve-healthcheck-canary/0.log" Nov 04 07:44:29.091432 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:29.091393 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g6lhj_7dcd4bbc-400c-4fce-b87c-0c018fadecfa/kube-rbac-proxy/0.log" Nov 04 07:44:29.192239 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:29.192189 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g6lhj_7dcd4bbc-400c-4fce-b87c-0c018fadecfa/exporter/0.log" Nov 04 07:44:29.300998 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:29.300970 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-g6lhj_7dcd4bbc-400c-4fce-b87c-0c018fadecfa/extractor/0.log" Nov 04 07:44:29.376716 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:29.376687 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-d7xzv_26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3/discovery/0.log" Nov 04 07:44:29.422724 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:29.422684 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589fcc7ff4-dcwx7_ebcd9bc5-c463-4550-820d-6648936b12a3/kube-auth-proxy/0.log" Nov 04 07:44:29.508749 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:29.508720 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-7wfxn_c3c860e0-8952-4f74-acd1-803a8a6acc85/istio-proxy/0.log" Nov 04 07:44:29.533354 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:29.533326 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-bx8vz_e734faeb-28a7-4aa2-86cc-dc9c24cf5c34/istio-proxy/0.log" Nov 04 07:44:29.575484 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:29.575457 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-lhm7v_ae22c880-563f-4c2c-9461-b1e185adf358/istio-proxy/0.log" Nov 04 07:44:29.716444 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:29.716367 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-7ppvl_27d5958b-d699-4846-b674-3bf2ef6d25ca/payload-pre-processing/0.log" Nov 04 07:44:29.741055 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:29.741027 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wzjpf_4522cb36-2dcc-483d-bf1e-42ffd1029cee/payload-processing/0.log" Nov 04 07:44:30.886218 ip-10-0-131-6 kubenswrapper[5385]: E1104 07:44:30.886172 5385 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-g6lhj" containerName="extractor" Nov 04 07:44:31.388138 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:31.388104 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-d7xzv_26d70d1d-8ef8-4caf-942d-5c6efb7b3fe3/discovery/0.log" Nov 04 07:44:31.420835 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:31.420810 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-589fcc7ff4-dcwx7_ebcd9bc5-c463-4550-820d-6648936b12a3/kube-auth-proxy/0.log" Nov 04 07:44:31.534428 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:31.534399 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-7wfxn_c3c860e0-8952-4f74-acd1-803a8a6acc85/istio-proxy/0.log" Nov 04 07:44:31.572369 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:31.572336 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-bx8vz_e734faeb-28a7-4aa2-86cc-dc9c24cf5c34/istio-proxy/0.log" Nov 04 07:44:31.620634 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:31.620605 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-74854d8ccf-lhm7v_ae22c880-563f-4c2c-9461-b1e185adf358/istio-proxy/0.log" Nov 04 07:44:31.744148 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:31.744081 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7b98bb678c-7ppvl_27d5958b-d699-4846-b674-3bf2ef6d25ca/payload-pre-processing/0.log" Nov 04 07:44:31.766831 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:31.766806 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7bb9489676-wzjpf_4522cb36-2dcc-483d-bf1e-42ffd1029cee/payload-processing/0.log" Nov 04 07:44:32.192795 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:32.192762 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-8f94f88d4-2xfwc_ce14b071-5b23-4026-ae18-122fc62ebee8/manager/0.log" Nov 04 07:44:32.280759 ip-10-0-131-6 kubenswrapper[5385]: I1104 07:44:32.280730 5385 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-8f94f88d4-2xfwc_ce14b071-5b23-4026-ae18-122fc62ebee8/copy-manifests/0.log"